When Alcoa Inc.'s senior attorneys sat down in late 2004 for their annual review of outside counsel costs, they wanted to get a feel for how their peer group's expenses compared.
The team first scrutinized data from off-the-shelf law department benchmarking surveys, but found that wasn't helpful because the universe of companies with more than $15 billion in revenue was too small. So the team hired Rees Morrison, co-chair of legal department consulting at Hildebrandt International, to conduct a benchmarking survey.
Second, acknowledge that like all statistical data, benchmarking metrics can be misinterpreted and misused.
"Benchmarking is often done to justify the status quo," says Aileen Leventon, partner in Blaqwell Inc., a legal industry consultancy. "The numbers are analyzed and presented to support a point of view, while data indicating a contrary point of view is ignored."
If you decide to use a publicly available survey, you'll want to take a close look at the range of products available. Prices top out at $3,500 (with discounts for survey participants) for the Hildebrandt International U.S. Law Department Survey, which covers the broadest range of metrics including staffing, spending, organization, outside counsel management and compensation data. Other surveys that focus on one or two of those areas are generally priced from $500 to $1,000. Magazine surveys are free to subscribers but have less rigorous methodology.
"You can always attack benchmarking data methodically," says Morrison, who literally wrote the book on law department benchmarking, "Law Department Benchmarks: Myths, Metrics and Management," published by Glasser LegalWorks. "You can't get enough participants to make correlations. Typically, you will have six to eight companies in a sample [for a tailored study.] It's not statistically reliable, but it will give you an idea of where you stand."
Because the number of respondents will be small, selection of the target group is key, and of that group, expect at best a 50 percent response, Morrison says.
Jonathan Bellis, co-chair of law department consulting at Hildebrandt International, who has directed what is now the Hildebrandt survey since it was first developed at PriceWaterhouse in the mid-1980s, says the key is viewing benchmarking as just one piece of the law department management puzzle.
"Benchmarking is a tool, not a panacea," he says. "Many tools should be considered. It's certainly a key thing that should be done, but it's not the only thing. You start with the survey data, but pretty quickly you need to get to best practices, based on an awareness of innovation. Otherwise, it's an empty discussion."