Innovation Blog

Before You Measure:  Define the Behaviors You Seek, OR

The Case of the Holey Azerbaijan Kettles

 By Shlomo Maital

  In this week’s Business Week, Lisa Hershman defines “the seven sins of performance measurement”. She raises a key point that has worried me for a long time.  We management educators teach that management begins with measurement.  What you cannot measure, you cannot manage.  This is mostly true. 

   The problem is,  how you measure things drives the behavior that creates the measure.  If you are not certain about which behaviors you seek to encourage, and if you are not certain about the link between your performance measures and behavior,  best not to measure at all.

   For example:  President Bush’s No Child Left Behind Act, the first piece of legislation he initiated after becoming President in 2000,  focuses heavily on performance testing in schools.  The result: Teachers teach how to do well on tests, not how to learn. 

    Here is the sad story of the Holey Azerbaijan Kettles, another example (no, Holey is not a mis-spelled word, it is indeed “holey”, based on “hole”). 

    In the old Soviet Union, factories were given quotas.  A kettle factory in Azerbaijan was given a quota of  “X” kettles to produce in the coming year, with the Gulag awaiting the plant manager if he failed to deliver.  Result: 100,000 kettles on schedule…but barely big enough to boil one cup of water.  Next year the performance measure defined the SIZE of the kettles. Result:  The plant manager conserved aluminum by making the walls of the kettles very thin, so thin they quickly developed holes.   

If you introduce performance measures to your innovation system, think very very carefully at the outset about which creativity behaviors you seek to foster and which measures will help you achieve this behavior.  If you are uncertain – best to forget it.

    Here, for the record, are the 7 sins of performance measurement, a la Ms. Hershman. 

                                   The Seven Sins of Performance Measurement

 1. Vanity:   using metrics that you have mastered, or that make you look good, rather than help drive improved performance results.  

2. Provincialism:   when you let organizational or department boundaries such as budgets determine your metrics.  Chinese GDP data are provided mainly by provincial governors. Guess why some numbers look super-terrific!

3. Narcissism:    when a company measures performance from its point of view, rather than the customer’s perspective.  (I think this is the worst, and most common, of all the sins).

4. Laziness:   Simply assuming what’s important to measure rather than taking the trouble to discover the right metrics by asking the clients.

5. Pettiness: Companies too often focus on a small part of what matters, rather than on the totality, or end results.  (At BP – reducing OSHA-defined accidents [OSHA is the US environmental protection law)] instead of truly implementing safety).

6. Inanity:   Creating metrics without any consideration for the consequences. (E.g. the Azerbaijan kettle).  Or:  a fast-food chain that chose to measure waste, based on how many cooked chickens went unsold at the end of the day. So managers responded by not cooking any chicken until orders were placed, thereby turning fast food into slow food.

7. Frivolity:  There are companies that just aren’t serious about metrics at all.  Any company that doesn’t take the business of measuring performance seriously won’t have anything to measure after a while.