Metrics_ How to Improve Key Business Results - Martin Klubeck [46]
No matter how you decorate it, metrics are only indicators and as such should elicit only one initial response: to investigate.
Of course, some metrics are simple enough that you will accept their story without as much investigation (like the gas gauge on your car), but even in these instances, you should keep a watchful eye in case they start to show you data that you believe is misleading or erroneous.
At the end of the day, even if you have total confidence in the accuracy of the data (pro-sports statistics, for example), you have to treat it all as indicators. Data can’t predict the future. If it could, then there would be no reason to play the games!
The point is that metrics should not be seen as facts but rather as indicators of current and past conditions. Used properly, metrics should lead our conversations, help us to focus, and draw our attention in the right direction. Metrics don't provide the answers; they help us ask the right questions and take the right actions.
Indicators: Qualitative vs. Quantitative Data
The simple difference between qualitative and quantitative data is that qualitative data is made up of opinions and quantitative data is made up of objective numbers. Qualitative data is more readily accepted to be an indicator, while quantitative data is more likely to be mistakenly viewed as fact, without any further investigation necessary. Let’s look at these two main categories of indicators.
Qualitative Data
Customer satisfaction ratings are opinions—a qualitative measure of how satisfied your customer is. Most qualitative collection tools consist of surveys and interviews. They can be in the form of open-ended questions, multiple-choice questions, or ratings. Even observations can be qualitative, if they don’t involve capturing “numbers”—like counting the number of strikes in baseball, or the number of questions about a specific product line. When observations capture the opinions of the observer, we still have qualitative data.
Many times, qualitative data is what is called for to provide answers to our root question. Besides asking how satisfied your customers are, some other examples are:
How satisfied are your workers?
Which product do your customers prefer, regular or diet?
How fast do they want it?
How much money are your customers willing to pay for your product or service?
When or at what hours do your customers expect your product or service to be available?
Do your workers feel appreciated?
No matter how you collect this data, they are opinions. They are not objective data. They are not, for the most part, even numbers. You can take qualitative data and try to transpose them into more quantitative forms—turning opinions into values on a Likert scale, for example. But in the end, they are still opinions. They’ll look like quantitative data, but they are not.
Some analysts, especially those that believe the customer is always right, believe that qualitative data is the best data. Through open-ended questions these analysts believe you receive valuable feedback on your processes, products, and services. Since the customer is king, what better analytical tool is there than to capture the customers’ opinion on your products and services?
These analysts love focus groups and interviews. Surveys will suffice in a pinch, but they lack the ability for analysts to observe the non-verbals and other signs that can help them determine the answer to the question, “How satisfied are our customers?”
One of the most popular organizational development books in recent years is First, Break All the Rules by Marcus Buckingham and Curt Coffman (Simon & Schuster, 1999). This book