Be Careful What You Measure: The Dark Side of Business Metricsposted by John Spacey, February 04, 2011
Business metric disasters usually start with the best of intentions.
It is natural for companies to seek business insight with metrics. Once those metrics are defined it is also natural to use them to assess the performance of business units, teams and individuals.
The problem is that evaluating people based on metrics changes their behaviour — often in unexpected and counter-productive ways.
The perils of partial metricsIt is not always easy to implement metric measurement. Often projects are required to introduce new technologies and processes.
Many companies take a phased approach to metrics. Often the first project involves measuring the two or three metrics that are most important to enterprise goals and strategies.
There is nothing wrong with this approach in itself. However, it is critical to have complete metrics in place before you start measuring performance based on those metrics. Metrics don't need to be complete across the enterprise — but they do need to be complete for the area being evaluated. If they are not overall business performance may decline as resources over-focus on areas being measured.
Example 1: call centreCall centers were amongst the first to measure staff according to low level performance metrics.
Consider a call center that measures just one metric: average call time. The result may be declining customer service as staff seek to minimize call times. It may even be in staff's best interests to hang up on demanding customers.
Now lets say the call center tries to fix the situation by applying one additional metric: customer satisfaction after the call. They begin asking customers to do a survey after each call.
The result may be that staff begin telling customer what they want to hear — even if it is misinformation ("yes sir, the technician will be to your house between 9-10 tomorrow morning"). Worse, staff may begin giving customers anything they ask for such as discounts or account credits. The result may be decreased profit margins, increased costs etc...
Example 2: network support techniciansMetrics may cause departments, teams and staff to over-focus on that which is being measured.
If you measure your network support technicians on Mean Time To Repair (MTTR). They may start:
1. Requiring people to open support tickets before they answer simple questions — breaking down communication.
2. Avoiding complex problems.
3. Applying band-aid solutions rather than true fixes.
4. Neglecting customer service and personal relationships.
5. Putting less effort into documentation, user training and other responsibilities.
Likely MTTR will improve: but it is unclear whether this translates to greater business value.
Gaming the systemWhen staff are measured according to automatically calculated metrics they will over focus on those metrics. Worse, they may start looking for ways to game-the-system.
Staff may begin to see their job as a video game where the goal is to score points by any means necessary.
In order for metric based evaluation to be successful metrics must measure total performance.
Quality is to business what gravity is to the universe. But what is quality? |
The words strategic risk strike fear into the hearts of investors and executive leaders everywhere.|
There's often confusion about the difference between Responsible and Accountable. Even seasoned program managers sometimes confuse the two. Here's the difference.|
The following definitions look at quality from a management, quality assurance, product, marketing, manufacturing and economic point of view.|