Metrics_ How to Improve Key Business Results - Martin Klubeck [90]
Figure 9-8. Customer satisfaction rating score as an average
So, the average score lacked meaning and comparing the highly satisfied to the not satisfied was a little confusing. A third choice was to use the percentage satisfied. I could understand quickly that a certain percentage of our customers (those who used our service) were either satisfied (4 or 5) or not satisfied (1, 2, or 3). Even with this I received arguments about the nuance of the meaning of “not satisfied.” I had more than a few managers who wanted 3s not to be counted since they thought on a 5-point scale that 3s were neutral.
I had to explain that “neutral” meant the respondent, while not “dissatisfied” couldn't say “satisfied” either. The chart wasn't comparing satisfied to “dissatisfied”—but satisfied to “not satisfied.” Notice that the same managers who wanted to include neutral scores in the first ratio (5s to 1–3s) wanted to drop neutral scores if they thought it would make their department look worse. Figure 9-9 shows why “percentage satisfied” was a simpler way to interpret the data.
Figure 9-9. Percentage of satisfied customers
We were on a pretty steady streak of adding measures to the original plan, so we had no reservations when it came to customer satisfaction. We realized that we were only looking at feedback from those who had problems. This could give us a skewed view of our customers' overall satisfaction with our services. We would never hear from customers who wouldn't call our Service Desk because they didn't like our services. Or we could miss those who liked our services, but either didn't choose to fill out a survey or just hadn't used it in the current year. Basically, we wanted to hear from customers who hadn't called into the Service Desk. We wanted to hear from the rest of our customer base, which was not reflected in our Usage measures.
The answer was an annual customer satisfaction survey. We sent a survey that not only asked the basic questions about satisfaction with our services, but also which services were seen as most important to the customer. This helped with the other services we included in the report card. We also asked the “who is the preferred source for trouble resolution” question, which we used for Usage. This annual survey provided many useful measures besides customer satisfaction. Table 9-8 shows the first breakout of data for this category.
For the annual survey we were able to show the same percentage satisfied, but pulled from a different context. This may not seem too important, but it was very useful to allow for different viewpoints. One telling result was that the scores from the annual survey were considerably “lower” than those for trouble resolution. This flew in the face of what the departments expected (this rang true for all of the services). The staff incorrectly predicted that the scores for trouble resolution would be worse than those for annual surveys. They figured that customers filling out the trouble-resolution surveys were predisposed to be unhappy since they had a “problem,” whereas the annual survey had a good chance of catching the customer in a neutral or good mood.
But the data proved them to be totally wrong. The resolution survey scores were significantly higher than the annual survey. This led to further investigation to determine why there was such a drastic difference (and one that went against the predictions). The investigation wasn't intended to improve the annual survey numbers—it was intended to provide understanding and, from that understanding, possible ideas.
One conclusion was that the resolution was done so well, (fast, accurate, and with a high rate of success) that the customer was so pleased that he gave great ratings. Conversely the annual survey reflected simply ambivalence at the