Metrics_ How to Improve Key Business Results - Martin Klubeck [125]
Competitor Availability: (1,440 – 240) ÷ 1,440 = 83.3%
If you had all of the raw data for your competitor's reports, you could use your personal “standard” and determine how well you perform against your competitor. But this is highly unlikely to happen. What you will get, if you are extremely lucky, is the “score”—and even that can be difficult to get from your peers and competitors.
So, how you define an outage compared to the way your competitor defines an outage is critical to your use of their measures as a benchmark.
Standards Allow Comparison to Others
Standards in performance measurement come down to providing the ability to compare measures between different organizations. Just as manufacturing standards allow you to use your products seamlessly with another organization's products, standards in performance measurement allow you to “use your measures” seamlessly with another organization's measures. If there were standardization of performance measures, you could “borrow” another unit's measures for your own purposes. If you had the same questions, you could trust that that metrics used by Company A could be used to measure the same things in your organization.
Without standards, it is hard to imagine how you could use the metrics of a different organization—even if you had the same exact root questions. And using measures produced by another organization as a benchmark is even more improbable.
Getting Good Data
This problem with benchmarks and standards for performance measurement also creates problems for well-meaning organizations that seek to provide data warehouses of information. This information invariably is intended for your comparison. To make the data warehouse effective, it has to have a clear set of standards for the information provided by the different organizations.
HDI, a third-party survey company, offers benchmarking on customer satisfaction data by controlling data definitions. HDI administers the survey to your customers, collects the responses, and provides you with reports, analysis, and raw data. Since HDI standardizes its questions (the same questions are asked along with the same set of possible replies on a 5-point Likert scale), it is able to offer you comparisons against other organizations, including the following:
Comparison of your scores (average, percentage satisfied, or other) against all other organizations who have used the HDI service
Comparison of your scores with others in your industry (self-selected from a list)
Comparison of your scores to the top nth percentile of others' customer base
This provides you with a higher confidence in the comparability of the information. Of course, there are some drawbacks. Only organizations that use HDI's service are included in the comparison, and your main competitors or peers may not be among these organizations. Even if you compare your results against the entire customer base, this still may not reflect the pool you want to compare to. Another minor drawback is that you are forced to use one set of questions. No deviations. If you want to use the 10-point scale suggested by Reichheld in The Ultimate Question, you could not use HDI's service. And even if you could use the 10-point scale, you could not compare the data to those who used a 5-point scale.
The Goal: Reliable Industry Standards
Industry standards for performance measures would make it possible to truly benchmark, rank, and compare peer organizations. It should be feasible to convince an industry (like higher education IT) to standardize performance measures before the chance of adopting universal standards. The tighter you can make the pool for standardization, the easier it should be to come to agreement. Higher education information