• Choose a category
• All
• Classic-Fiction

The use of triangulation and expectations were critical to the success of the program—partly because it gave a better picture of the answers to the root question and partly because they helped to remove much of the fear, uncertainty, and doubt that normally accompanies metrics.

Conclusion

Now that I've laid the groundwork for the Report Card, it's time to finish the effort. I needed a way to make the “big” picture the groundwork supported meaningful for the director's level, the CIO, and beyond. If the organization was to publish the metrics for its upper leadership, it would also have to be ready for other audiences, including shareholders or stakeholders. Customers might also want to see results of how well the organization serves them.

This requires that the metric be easily modified to show different views for different audiences. Of course, it also had to be meaningful to the service provider.

Final Product: The Metrics Report Card

Chapter 9 introduced the metrics I developed to answer the leadership's question for our organization. If only the service provider and the executive leader were to be viewers of the results, I could have moved directly to publishing the metric. But since the metrics would go through rigorous review throughout the management chain, and also be seen by customers, I had to find a way to make the results usable (if not readable) at each level.

I worked with a team of consultants and the service providers to develop what became the Report Card. This means of interpreting the metrics allows for it to be viewed at different levels, and at different degrees of aggregation, while still preserving the concept of expectations.

Now that basic measures were developed, we (the team of consultants— the service providers and me) needed a way to make them into a Report Card. What we had was a set of charts. But we needed to find a way to report on the overall service health, while still preserving the individual measure categories of Delivery, Usage, and Customer Satisfaction. Being at an academic institution, it seemed logical to use the Report Card concept as a template.

In the case of a Report Card, students can take totally disparate courses—everything from forensic anthropology to fine art printmaking. The grades obtained can be based on totally different evaluation criteria, but the grades are still understandable. An A means the student is exceeding expectations, doing very well; a B means doing well; a C means the student is surviving; below a C means the student is failing to meet expectations. Quizzes, tests, research papers, and classroom participation can be used to make up the grade. Other less normal evaluation methods can be used—like reviews of art produced, presentations to committees, and panel reviews of materials produced in the course of the class.

In all cases, the student gets a letter grade that can be transferred to a number for grade point averages. We wanted something similar that a high-level stakeholder could glance at and grasp immediately.

We settled on E, M, and O. We had to figure out how to evaluate and determine each measure and piece of information as either Exceeding Expectations, Meeting Expectations, or an Opportunity for Improvement. Availability was in the form of percentage (of abandoned calls or abandoned calls less than 30 seconds). So we could smoothly transition this to a common measurement view.

Delivery

Remember, for this discussion I've broken delivery into “availability,” “speed,” and “accuracy.”

Availability

For each measure, expectations have to be identified. Table 10-1 shows the expectations for Availability.

This made the results simple for analysis. Placing the measures into the grid would give us a “grade” of O, M, or E. This could then be used to develop a grade point average. We could roll up the Grades at the Availability level. We could also roll up the grade at the Delivery level (Availability, Speed,