• Choose a category
• All
• Classic-Fiction

As I've already covered, you have to work hard to get to the right root question: develop an abstract picture of the answer, identify the information needed to paint the picture, and then painstakingly set up processes for collecting, analyzing, and reporting the metric. And through all this, you have to double- and triple-check everything from the data to the collection methods to the metric itself, and then, finally, the root question.

Be diligent and rigorous in your efforts because it is extremely easy to make errors. Even when all of your data are verified, you can have errors in interpretation. I once had an interesting debate with a coworker over the concept of facts. He felt that metrics, at least good ones, were facts; and if they weren't facts, we shouldn't use them to demonstrate performance. I had to explain, at least from my view, that metrics are not facts. Metrics are first and foremost indicators. They give us insight, but they are not necessarily the truth that is being sought. They are not facts.

Metrics are not facts. They are indicators.

Metrics: Indicators or Facts?

This distinction as to whether metrics are indicators or facts is at the core of proper metrics use. If we treat metrics as facts, we run the real risk of making decisions too hastily.

How about measures or data? Is the speedometer on your car relaying facts about your speed? Is it precisely accurate or does it have a +/- deviation? If your speedometer says your going 55 and the police radar says you're going 58, which is truth?

You may argue that while there are variances in measuring devices, there is obviously a true speed you were traveling at. And, I'd agree. You were definitely traveling at a specific speed at a given moment. But, I have little faith that any device used to capture a particular moment in time is accurate enough to call that measure a “fact.”

Let's try subjective measures. Say I ask you to rate your satisfaction with my service on a scale from 1 to 5, with 1 being highly dissatisfied, 2 being dissatisfied, 3 being neutral, 4 being satisfied, and 5 being highly satisfied. I should be able to consider your choice to be a fact, right?

Wrong.

The only fact that I can ascertain on a survey is that the answer I receive is the answer you gave. And even then we may have errors. In customer satisfaction surveys, we often find that respondents get the numbers inverted and give 1s when they meant to give 5s. Barring this type of error, can't we say the results are facts? Again, the only thing we can categorically attest to is that the answer we have is the answer the respondent chose. We cannot know for a fact that the answer given was the true answer.

This uncertainty has been analyzed and researched to the point where I can say with confidence, that most answers are actually not true. In The Ultimate Question(Harvard Business Press, 2006), Fred Reichheld researched the best customer satisfaction questions to ask to determine potential business growth. His study was based on responses from promoters (those who would recommend a product/service) and detractors (those who would steer people away from a product/service). One by-product of this effort was the realization that people don't answer surveys in a totally truthful manner.

Basically, Reichheld found that on a 10-point-scale question, a “6” is not truly neutral. Most people who felt neutral about the product or service being rated actually gave 7s or 8s, although this range was clearly marked as being more favorable than neutral.

I believe this happens on a 5-point scale also. Most customers don't want to give you a “3” if they feel ambivalent about the product or service. Let's look at a simple translation, shown in Table 14-1, which I propose is much closer to the truth for the majority of respondents of a customer satisfaction survey.

What I've found is that unless the respondent was actually angry about the service, he won't give it a “1.” Therefore, 2s become the choice of the very dissatisfied (those just short of angry).