Morrison on Metrics: Synthetic Indices

What some people call "synthetic indices" combine (synthesize) several related measures into a single number - the index. For example, everyone knows the Consumer Price Index: The aggregated and weighted prices of a basket of common goods and services.

In the law department context, to illustrate, we could envision a synthetic index of "corporate IP prowess," constructed from (1) the number of patents applied for in a year, (2) R&D spending as a percentage of revenue, (3) the number of references in the annual report to innovation and intellectual property, and (4) licensing revenue from patents. For a group of companies, if you could collect those four related measures from each of them, put them on a comparable scale such as deviation from the average, and sum them, you would create an index of corporate IP prowess.

I created a different index with my benchmark rankings of industries and countries. I call it an index of legal intensity. With four key benchmark metrics I fashioned an overall ranking (an index) for the industries and countries. Email me if you would like a copy.

General counsel could create and encourage indices for many of their operational concerns, such as engagement of employees and talent, which could be measured by teamwork, diversity and camaraderie. Or think of the possibilities with contract complexity or outside counsel usage.

What can you do with a synthetic index? Of immediate use is to help you understand a complicated situation with some quantification. Related to that, with several participants you can benchmark yourselves against the median figure. Then too, a thoughtful index with reliable numbers resembles a dashboard for something important. Individual components can be shifted but the total set - the index - is harder to alter. Alternatively, once you have an index, you can show how other metrics correlate to that index. Keeping with IP prowess, is there a relationship between where a company ranks on it total legal spending? If you want to add sophistication to an index, you can assign different weights to the components. If number of patents granted seems much more significant, for instance, you can double its contribution to the overall score.

You do have to be careful with synthetic indices. Among the methodological difficulties, those who interpret relative standing may ignore margins of error: they regard tiny differences between rankings as significant. Also, minor changes in components can have magnified effects on the outcomes. An irregular surge one year in licensing revenue from patents would disproportionately alter the index. Another problem arises if the narrow components are mistakenly treated like a proxy for a broader measure. IP prowess might best be measured by the percentage of revenue that comes from products or services less than five years old.

By the way, a second use of the term "index" is when someone puts every metric on the same basis, such as is done by the Economist's Big Mac index. It converts the local price of a Big Mac to the prevailing exchange rate and shows whether the currency is relatively over- or under-priced. A third meaning of "index" is as a mathematical formula. For example I have written on my blog about Herfindahl's index of industry complexity, an entropy index of diversification and a general-purpose heterogeneity index.

Synthetic indices, when used skillfully, can help us understand and quantify a domain better than we did before.

Join the Conversation

Advertisement. Closing in 15 seconds.