Metrics_ How to Improve Key Business Results - Martin Klubeck [129]
If you discounted all 3s as neutral responses, you may be ignoring a large contingent of dissatisfied customers.
So, what metric is fact? Especially in our definition of a metric, which is made up of multiple data, measures, information, and even at times other metrics. There are enough variables at this level to make any answers way short of “fact.” How about at the lowest levels, though? How about data? Can't it be trusted to be factual?
No. Scientists keep finding that things they knew to be a scientific fact yesterday are totally wrong today. Automated data collection systems can easily be miscalibrated and provide erroneous data (my bathroom scale is constant proof of this). When we add people to the equation, the possibility of errors increases.
Technology is great; as it advances, the accuracy of data increases. But even when using current technology, it is critical not to treat metrics as facts. When you give metrics more weight or significance than they deserve, you run the risk of making decisions based solely on the data.
Although base information should never be considered to be entirely factual or without fault, it shouldn't deter you from the proper use of metrics. But hopefully this knowledge will guide you to use metrics as they are intended to be used—as indicators to help support your decisions.
Metrics should never replace common sense or personal involvement.
Misused Metrics: “Our Customers Hate Us”
Let me provide a real life example of how a manager's well-intentioned use of metrics did more harm than good. A team of hard workers were told that they were hated by their customer base. Or at least that's how they interpreted the story shared with them by their boss.
Every two weeks, the CEO would meet with his department heads. For the first 30 minutes of the meeting, they'd review every customer satisfaction rating of a 1 or 2 (out of 5) across the organization. These ratings were labeled highly dissatisfied and dissatisfied.
In reality, all this exercise showed was the number of respondents who chose a 1 or 2 rating. We don't know much more than that.
The comments with each rating, when given, were also scrutinized. Based on these comments, most customers were clearly unhappy, but occasionally it was obvious that the respondent simply picked the wrong rating.
Looking at each case, it was clear to me that most customers only gave a 1 when they were angry, and they always used the opportunity to give a lengthy comment on why they were upset.
After the department heads reviewed the surveys with 1s and 2s, if time allowed, they would look at 3s since the comments provided normally indicated a level of dissatisfaction and pointed toward areas that could be improved.
This review was well-intentioned. The company leaders were, after all, listening to the customer's voice. That's why they administered the surveys and reviewed the negative responses.
For each survey response, the following needed to be explained:
Why the low rating was given. If a customer's comments weren't clear (or there were no comments) someone on staff should have contacted the customer for clarification.
What was done to “make it right” with the customer.
What could have been done to avoid the low rating. (A better way of phrasing this would be, “What could we have done to prevent customer dissatisfaction?” A nuance, but important. If we tie our improvements to the measure rather than the behavior or process, we run the risk of improving the numbers without changing the behavior or process.)
Unfortunately, the last item—how to improve so that the customers are not dissatisfied in the future—received little attention in these meetings. This isn't uncommon, however. I've seen leadership demand explanations of why customers were dissatisfied, but the goal should be to improve processes to eliminate repeat occurrences.