Metrics_ How to Improve Key Business Results - Martin Klubeck [43]
One good argument on the topic of old facts not being in line with new truths is that facts don’t change, just our interpretation of them.
Let’s take a quick look at a fact that perhaps isn’t truth.
It concerns the Amazon book sales rankings. Michael Langthorne, one of my former coauthors, enjoys watching the sales ranking of our book on Amazon.com. He has a high level of confidence in the data and his level of excitement grows or wanes based on the numbers. The problem is that the rankings are daily and depend on not only the number of sales of our book, but also the number of sales of all the other books on the site. The sales are also only counted for individual buyers. If you were to buy a thousand copies of our book, Amazon would only count that as one sale toward the rankings. If you were to instead make a thousand separate orders, it would then be counted as one thousand sales, boosting the rankings. Another issue is that Amazon doesn’t care who purchases the book. If I were to buy those books, it would improve our rankings, although with no bias because I’m one of the authors.
The point is simple. While the data is “accurate” (or at least you can have a high confidence level in them being accurate), the interpretation of that data can be problematic. Should Mike buy a new television in anticipation of rising sales due to the increased popularity of the book? Or should he be depressed over the lack of sales if the rankings fall drastically?
It’s fairly obvious that the answer to both questions is no.
This misrepresentation of metrics as fact can be seen in instances where only a portion of the metric is relayed to the viewer.
A business example is one a friend of mine loves to tell about the service desk analyst who was by all accounts taking three to five times as long to close cases as the other analysts. The “fact” was clear—he was less efficient. He was closing less than half of the cases as his peers and taking much longer to close each case. His “numbers” were abysmal.
The manager of the service desk took this “fact” and made a decision. It may not have helped his thought process that this “slow” worker was also the oldest and had been on the service desk longer than any of the analysts. The manager at the time made the mistake of believing the data he was looking at was a “fact” rather than an indicator. And rather than investigate the matter, he took immediate action.
He called the weak performer into his office and began chewing him out. When he finally finished his critique he gave the worker a chance to speak, if only to answer this question (veiled threat): “So, what are you going to do about this? How are you going to improve your time to resolve cases? I want to see you closing more cases, faster.”
Showing a great deal more patience than he felt at the moment, the worker replied, “My first question is, how is the quality of my work?”
“Lousy! I just told you. You’re the slowest analyst on the floor!”
“That’s only how fast I work, not how good the quality is. Are you getting any complaints?”
“Well, no.”
“Any complaints from customers?”
“No.”
“How about my coworkers? Any complaints from them?”
“No,” said the manager. “But the data doesn’t lie.”
“You’re right, it doesn’t lie. It’s just not telling the whole story and therefore it isn’t the truth.”
“What? Are you trying to tell me you aren’t the slowest? You are the one who closes the cases. Are you just incompetent?” The manager was implying that he wasn’t closing the cases when done.
“No, I am the slowest,” admitted the worker. “And no, I’m not incompetent, just the opposite. Have you asked anyone on the floor why I’m slow?”
“No—I’m asking you.”
“Actually you never asked me why. You started out by showing me data that shows that I’m ‘slow, inefficient,’ and now ‘incompetent.’”
The manager wasn’t happy with the turn this had taken. The employee continued, “Did you check the types of cases I’m closing? I’m actually