Proofiness - Charles Seife [4]
It’s true: all measurements are imperfect. However, some are more imperfect than others. As a result, not all numbers are equally fallible. Some numbers, those that are based upon extremely reliable and objective measurements, can come very close to absolute truth. Others—based on unreliable or subjective or nonsensical measurements—come close to absolute falsehood. It’s not always obvious which are which.
Truthful numbers tend to come from good measurements. And a good measurement should be reproducible: repeat the measurement two or ten or five hundred times, you should get pretty much the same answer each time. A good measurement should also be objective. Even if different observers perform the measurement with different kinds of measuring devices, they should all agree about the outcome. A measurement of time or of length, for example, is objective and reproducible. If you hand stopwatches to a dozen people watching the same event—say, the Kentucky Derby—and ask them to time the race, they’ll all come up with roughly the same answer (if they’re competent). A whole stadium full of people, each using different stopwatches and clocks and time-measuring devices, would agree that the race took, say, roughly one minute and fifty-nine seconds to complete, give or take a few fractions of a second. Similarly, ask a dozen people to measure an object like a pencil, and it doesn’t matter whether they use a ruler or a tape measure or a laser to gauge its length. When they complete their measurements, they’ll all agree that the pencil is, say, four and a half inches long, give or take a fraction of an inch. The result of the measurement doesn’t depend on who’s doing the measuring or what kind of equipment’s being used—the answer is always roughly the same. This is an essential property of a good measurement.
Bad measurements, on the other hand, deceive us into believing a falsehood—sometimes by design. And there are lots of bad measurements. Luckily, there are warning signs that tell you when a measurement is rotten.
One red flag is when a measurement attempts to gauge something that’s ill-defined. For example, “intelligence” is a slippery concept—nobody can nail down precisely what it means—but that doesn’t stop people from trying to measure it. There’s an entire industry devoted to trying to pin numbers to people’s brains; dozens and dozens of tests purport to measure intelligence, intellectual ability, or aptitude. (An applicant to Mensa, the high-IQ society, has his choice of some thirty-odd exams to prove his intellectual superiority.) Testing is just the tip of the multimillion-dollar iceberg. After measuring your intelligence, some companies sell you a set of exercises that help you improve your score on their tests, “proving” that you’ve become smarter. Dubious claims are everywhere: video games, DVDs, tapes, and books promise to make you more intelligent—for a price. Even the British Broadcasting Company tried to cash in on the intelligence-enhancement fad. In 2006, a BBC program promised that you can become “40 percent cleverer within seven days” by following diet advice and doing a few brain-teasers. Was there a sudden surge in the number of Britons understanding quantum physics? Unlikely. So long as researchers argue about what intelligence is, much less how to measure it, you can be assured that the “40 percent cleverer” claim is worthless. In fact, I can personally guarantee that you’ll instantly be 63 percent smarter if you ignore all such statements.
Even if a phenomenon has a reasonable definition that everybody can agree about, it’s not always easy to measure that phenomenon. Sometimes there’s no settled-upon way to measure something reliably—there’s no measuring device or other mechanism that allows different observers to get the same numbers when trying to quantify the phenomenon—which is another sign that the measurement is dubious. For example, it’s hard to measure pain or happiness. There’s no such thing as a painometer or a happyscope that can give you a direct and repeatable reading about what a subject