Metrics_ How to Improve Key Business Results - Martin Klubeck [138]
Research is not cheap. While technology makes the collection of data much easier than in the past, it's still costly. The monitoring software and hardware may be expensive. The use of manpower to analyze and report the data is always expensive.
While I strongly advise against conducting exploratory research, I fully support your taking every advantage of those who have the time, resources, and will to do so.
Recap
Here are some points worth highlighting:
Don't neglect defining your questions because someone can provide you with ready-made answers. Don't let existing research cloud your issues and keep you from asking the right questions. It's tempting to use others' research and not do the upfront work of determining your needs.
Don't let existing research push you in a direction that allows comparison at the expense of having meaningful data for your situation.
Don't let others dictate your metrics with the ambiguous “interesting.” Learn to say “no” to the “I think it would be interesting…” requests.
Don't rely on others to provide your questions—only you truly know your needs.
Don't give more weight to aggregate data than to your own metrics.
You may have noticed that the admonitions above all stem from the core methodology for developing metrics—first identify the root question. The biggest issue with non-directed research may be the lack of a meaningful question. If the research is being conducted by an independent entity, the question is usually generic, and therefore, meaningless for most.
An example would be research done on the high-school graduation rate for children from one-parent families. This “interesting” measure can garnish a decent grant from our government, but how useful is the information to your local high school? Besides the ability to say that high school X has a better graduation rate for their single-parented population than high school Y, how useful is this data? If the data shows that fewer children from single-parent homes graduate than those from two-parent homes, will the schools focus differently on those children because they are more at risk? It may be useful data for debates against single-parent child-rearing or for more support for these families. The bottom line is that while the research is interesting and may be worthy of funding, it is not highly useful for the local school. It has uses, but not for where the “rubber meets the road” in education.
At least the example given involves research for a purpose, albeit a vague one. The other type of research, which your organization is more likely to engage in, is much more problematic. The research conducted by your organization not only falls prey to the same lack of a meaningful question (usually it's conducted without forethought or planning but simply because someone thought it would be interesting), but wastes your resources! So, unless you have limitless resources (including time), I highly recommend against collecting, analyzing, or reporting any data that does not answer a specific and meaningful question.
Conclusion
Research has great potential for helping us find answers. The trick is to remember that the research conducted by third parties will rarely provide the direct (or full) answer to your questions. You definitely should leverage the work of others, but only after you've clearly defined your question.
The other key point to remember is that you likely don't have the resources to conduct non-directed research (leave it to those who receive grants for such things or are in the business of research). This may seem easy, but you may find yourself falling