Morrison on Metrics: Lies told by the equals sign

“Equal” rarely means what we think when we collect law department metrics

For the past 500 years, mathematicians and scientists have used symbols, the crucial one being the equals sign. Unusually, we know who invented it and why. Robert Recorde, in 1557, wrote in his treatise, “The Whetstone of Witte: "To avoide the tediouse repetition of these woordes: is equalle to: I will sette as I doe often in woorke use, a paire of paralleles, or gemowe lines of one lengthe: bicause noe .2. thynges, can be moare equalle." This wonderful bit of mathematical history, quaintly recorded, comes from Ian Stewart, writing in the Feb. 13, 2012 online issue of New Scientist.         

We owe thanks to Recorde because we use his sign or its text equivalent all the time. With law department metrics and benchmarks, it is a mainstay. What we often gloss over, however, are the simplifying assumptions that underlie many claims of “equality.”

When we collect and analyze figures on pending lawsuits, such as lawsuits per in-house lawyers, is one case equal to another when we count them? Hardly. One class action hugely outweighs all the little slips and falls. One antitrust case with treble damages looming puts all the commercial disputes over a breached contract in the shade. The equals sign papers over confounding differences.

Is one lawyer equal to another lawyer? Not if the talent people have it right. The productivity and quality of lawyers, even those the same number of years out of the same law school, can vary by magnitudes. We blithely refer to a “10-lawyer department” without acknowledging the enormous range of talent that it might house compared to another 10-lawyer department, notwithstanding the implied equals sign.

Even something as stable as a dollar of compensation frustrates the equal sign if you think in terms of purchase power parity. A dollar paid the lawyer in San Francisco disappears much more quickly than the dollar paid the lawyer in Austin.

In short, when we call things equal we often gloss over manifold differences. When we say that one department “is the same as” another on a benchmark metric, the proposition holds analytically—the two calculated figures match—but the underlying facts rolled up into the figures are likely to be distantly far afield. It is the plight of all metrics: To be a metric is to ignore detail among the constituents.

author image

Rees Morrison

Rees Morrison, Esq. is a partner at Altman Weil, Inc. with countless interests in legal data analytics. He is also the founder of General Counsel Metrics, LLC....

Bio and more articles

Join the Conversation

Advertisement. Closing in 15 seconds.