Is it bad for your law department to be above or below some average given by benchmark survey? For example, if the number of lawyers you have is six for every billion dollars of your company’s revenue (say you have 12 lawyers and are a $2 billion company), what if the survey average—or median—from the survey is 5.7 lawyers? What if some consulting firm recommends, based on the survey, that you reduce your lawyer headcount?

To answer these questions, consider these six points:

1. You should honor the survey data only if comes from companies in your industry. Obviously, you should find the names of the participating companies. I should emphasize that the only meaningful comparisons are your figures against benchmarks in your industry. And, industries shouldn’t be overly broad, as in the ultimate mashup to services, manufacturers and others.
2. You can’t answer that question about how you’re doing without knowing another benchmark or two. Continuing with our example, if you also are under the benchmark on paralegals and under on other legal staff, it may mean you are “lawyer heavy,” but otherwise you can’t be definitive about a conclusion.
3. You might want to drill down on the participants in your industry. If you are smallish in comparison to many of the participants, you should make a bit of allowance for the economies of scale the larger ones enjoy. You also should take into account how many respondents make up the benchmark set for the common-sense reason that if there are only a few, such as seven or eight, the resulting average or median could change quite a bit if even only one or two companies were added. Remember that your own department’s figures are included and, to that degree, influence the median toward your figure.
4. If you have more lawyers than the comparison industry benchmark, you might want to know whether the variance has statistical significance. In other words, 5.7 lawyers is the published number, but given a small sample size, anywhere from 5.4 to six lawyers per billion might be just as true.
5. It might be that you have a heftier in-house group of lawyers, but your spending on outside counsel comes in lower than the benchmark number. The point is that you can’t react to having six lawyers per billion against a benchmark of 5.7 unless you consider where you stand on related metrics.
6. Ideally, you also should obtain quartile figures. They will let you judge how extreme your variance is.

To conclude the point made here, if you are below some benchmark figure and feel comfortable about the characteristics of the survey that produced the figure, you are in no more certain shape. The figures should encourage you to consider cognate metrics and some of the methodological aspects described in this column.

Once you conclude that your variance is well-founded and material a difference you can turn to the next task: whether to do something about your department’s staff or operations, and what you might do.