Metrics_ How to Improve Key Business Results - Martin Klubeck [94]
When we had trouble identifying the expectations, we'd play the estimation game. I'd start with the obvious: What exceeds expectations?
Someone would respond, “I can't tell you what exceeds expectations…”
“OK. Let's work on it together,” I'd reply. The reluctance to offer an estimate has many possible causes. Luckily you don't have to eliminate the causes, just deal with the effects.
I asked questions to help get to an estimate: “Would no abandoned calls be above the customers' expectations?”
“You mean that they would always get through?”
“Sure. No waiting on hold.”
“They'd love that!”
“Sure,” I'd say, “but would they expect it?”
An analyst would say, “No. They know we aren't manned with enough people to do that.”
“So they wouldn't expect that level of service?”
“Nope.”
“How about getting through within 30 seconds?”
“Yeah, they should expect that.”
I'd ask: “But do they?”
“Well, yeah,” someone would say, “for the most part. They know sometimes it's busy.”
“So sometimes they expect that it will take longer than 30 seconds?”
“Sure, like on Mondays, or when a system is down, or we're installing new software.”
“OK,” I'd say, “So how often is that?”
“Maybe 10 percent of the time.”
I'd sum up: “OK, so far, we can say then that 90 percent of the time they expect to speak to an analyst in less than 30 seconds?”
“I'd say they'd be happy with that.”
Since we always want ranges (not targets or thresholds) I pressed for more. “So, would they be happy with 85 percent of the time? Or 80 percent? If they call ten times in a month, they should expect to talk to an analyst within 30 seconds eight of those ten times, and the other two times would take longer?”
“Sure…that'd be OK.”
“How about three out of four times? Would they still be happy or would they not be satisfied?””
“Well, they may not like that. I mean, it's close.”
Again I sum up: “OK. Let's say then that the customer would expect to get through to an analyst within thirty seconds, 75 to 90 percent of the time. Does that sound about right?”
“Sure.”
Eventually I got to numbers that fit expectations. Once we plotted those expectations we performed the litmus test explained in the chapter on Expectations. We did this with each of the measures. We looked at the data over the years and checked the expectations against the department's perception of how well the service was provided during that time period. They knew when they had had a bad month. The relationship between the measures and the expectations should reflect their independent assessment of the quality of the service for that time period.
If we go back to the Metric Development Plan, you see the need to identify the schedule for collection and reporting. These items are not addressed separately. While we were defining the measures to use, we worked on the expectations. While we worked on the expectations we also identified the frequency of reporting and the time span for evaluation. We opted for monthly across-the-board reports, with roll-ups to the calendar year. It could have as easily been weekly, quarterly or only calendar, academic, or fiscal year.
Recap
In our real-life example, you can see that even when a mandate is given to implement a metrics program, you can get to a root question to allow the effort to be driven by a foundational need. In the case given, I was lucky that the leadership's information need was easily interpreted as a Product/Service Health question. This led cleanly to the development of effectiveness metrics for each key service.
Being able to stay in the first quadrant of the Answer Key mitigated much of the risks with implementing a large-scale metrics program in an organization that had not successfully done this