If you can't manage what you can't measure, with equal force, what you measure can be manipulated. Everything that a general counsel chooses to count - the first step in any effort to create a performance metric - not only raises definitional issues, but also interpretational differences. The primal term "matter," for example, as in matters per lawyer or number of environmental matters, needs a workable definition for someone to tally them. Matter could mean more than four hours of work, issues related to a certain degree, important enough to be recorded or a threshold spend on outside counsel. But even a crisp definition oozes into plenty of interpretational uncertainty and ingenuity. Clever lawyers, if their bonus, promotions or responsibility depend on the number of their matters, find ways to boost the number.

So too with the number of patents applied for, the number of law firms retained, the percentage of fees paid other than on an hourly rate and savings extracted from law firm invoices. If it becomes important to be counted, it is vulnerable to being gamed. Numbers don't exist in a vacuum, waiting to be picked up like marbles. To some extent, all numbered items are socially created.

A broad benchmark metric, such as total legal spending as a percentage of corporate revenue, affords fewer opportunities for distortion, because it relies on many metrics over time and if one is distorted, the others reflect it. Put differently, if you fudge one number, another may also budge.

But the many components that make it up are far from undebateable. As a fundamental determinant of legal services provided, it yields to some pushing and pulling. General counsel can push work back to clients, set limits on contract review, let human resources and tax departments retain their own specialist legal counsel, pull in contract lawyers instead of employees and otherwise manage down that key metric. For counts of smaller things, such as contracts reviewed or domain names acquired, more opportunity exists for "adjusting" the numbers.

Ironically, at a time when performance metrics have taken on increasing importance for law departments and are more under fire, that same focus brings to the forefront more enticements to, shall we say, cook the books. As tax lawyers find loopholes, soldiers inflate KIA reports and salespeople contrive to maximize their commissions, so too law department managers and lawyers find room and reason to maneuver to their advantage even the most hardened numbers.

All is far from lost. It is possible to game the gamers with clear definitions, periodic calculations of ratios, data trends over time and a dose of thoughtful skepticism.

This unavoidable malleability of metrics hardly means that we should throw up our hands and abandon measurement or benchmarks. We are better off with numbers that flicker now and then than with total darkness. While that is true, we should recognize something like the Heisenberg uncertainty principle: Once a general counsel decides to promote a particular metric, it becomes vitally important to define what is to be counted and to be vigilant for creative ways to change that count.