Morrison on Metrics: The Theory Behind Anonymous Participants in Law Department Benchmark Surveys

In the first group of 445 participants in the General Counsel Metrics LLC global benchmark report, 43 of them asked to remain anonymous. They did not want the report to list them as a respondent, although they welcomed having their data included in aggregated results and getting their results.

Why did they request anonymity?

Some were not the general counsel, and they might have been concerned about their right to disclose proprietary information. If you are the assistant general counsel, a direct report to the general counsel, you might be unsure about whether you are authorized to disclose total legal spending. But you want to know about your department's comparative performance.

One or two were in the finance function. They might not want their brethren in legal to know they were checking on the legal function's performance. Jockeying within the company for power--and knowledge is power--is unseemly.

Some were general counsel who might be unsure whether their metrics would put them in a good light. The instinctive propensity of some risk-averse lawyers is to protect confidential information. If your company name isn't listed, no one can know much. No one within the department can beseech you for the metrics; no CFO or CEO can get wind of it and put you on the spot for how your spending compares to the spending of other companies in the industry.

I dug into the nameless participants a bit more. At least eighteen of the 43 are at non-U.S. companies. Outside the U.S., there is less familiarity with benchmark studies and more reluctance to disclose data, let alone a name. In the U.S. such studies have been available for two decades and general counsel are accustomed to trusting their data to third-party aggregators. So aside from levels of authority, concerns about proprietary leaks and internal politics, levels of cultural comfort caused some of the participants to want to fly under the radar.

Careful researchers check for representativeness bias. They compare their respondents' to the total pool and look carefully for indicators that the data they have suffers from biases. In the future I may look at whether the metrics of anonymous companies are any worse than the public metrics.

For the most part, however, general counsel partake liberally in benchmark efforts. Of the three other benchmark surveys that are available to the public, a total of 515 participated in their most recent reports. None of the general counsel of those 500-plus law departments minded being listed on the rolls of participants.

Many did more than one survey. In fact, I counted 53 that completed two of the surveys. More surprisingly, four law departments completed all three of them. It is possible that some of the participants also took part in one or more private surveys that are not available to the public and do not release the names of the participating companies. I conduct some of those bespoke benchmarking studies.

author image

Rees Morrison

Rees Morrison, Esq. is the founder of General Counsel Metrics, LLC. Based in Princeton, NJ, Rees has for the past 25 years consulted solely to...

Bio and more articles

Join the Conversation

Advertisement. Closing in 15 seconds.