Drunkard's Walk - Leonard Mlodinow [89]
For example, suppose we are skeptics and hypothesize that the student cannot accurately predict the results of coin tosses. And suppose that in an experimental trial she predicts the coin tosses correctly in a certain number of cases. Then the methods we analyzed in chapter 4 allow us to calculate the probability that she could have accomplished the predictions by chance alone. If she had guessed the coin-toss results correctly so often that, say, the probability of her being that successful by chance alone is only 3 percent, then we would reject the hypothesis that she was guessing. In the jargon of significance testing, we would say the significance level of our rejection is 3 percent, meaning that the chances are at most 3 percent that by chance the data has led us astray. A 3 percent level of significance is fairly impressive, and so the media might report the feat as new evidence of the existence of psychic powers. Still, those of us who don’t believe in psychic powers might remain skeptical.
This example illustrates an important point: even with data significant at, say, the 3 percent level, if you test 100 nonpsychic people for psychic abilities—or 100 ineffective drugs for their effectiveness—you ought to expect a few people to show up as psychic or a few ineffective drugs to show up as effective. That’s one reason political polls or medical studies, especially small ones, sometimes contradict earlier polls or studies. Still, significance testing and other statistical methods serve scientists well, especially when they can conduct large-scale controlled studies. But in everyday life we don’t conduct such studies, nor do we intuitively apply statistical analysis. Instead, we rely on gut instinct. When my Viking stove turned out to be a lemon and by chance an acquaintance told me she’d had the same experience, I started telling my friends to avoid the brand. When the flight attendants on several United Airlines flights seemed grumpier than those on other airlines I’d recently flown with, I started avoiding United’s flights. Not a lot of data there, but my gut instinct identified patterns.
Sometimes those patterns are meaningful. Sometimes they are not. In either case, the fact that our perception of the patterns of life is both highly convincing and highly subjective has profound implications. It implies a kind of relativity, a situation in which, as Faraday found, reality is in the eye of the beholder. For example, in 2006 The New England Journal of Medicine published a $12.5 million study of patients with documented osteoarthritis of the knee. The study showed that a combination of the nutritional supplements glucosamine and chondroitin is no more effective in relieving arthritis pain than a placebo. Still, one eminent doctor had a hard time letting go of his feeling that the supplements were effective and ended his analysis of the study on a national radio program by reaffirming the possible benefit of the treatment, remarking that, “One of my wife’s doctors has a cat and she says that this cat cannot get up in the morning without a little dose of glucosamine and chondroitin sulfate.”8
When we look closely, we find that many of the assumptions of modern society are based, as table moving is, on shared illusions. Whereas chapter 8 is concerned with the surprising regularities exhibited by random events, in what follows, I shall approach the issue from the opposite direction and examine how events whose patterns appear to have a definite cause may actually be the product of chance.
IT IS HUMAN NATURE to look for patterns and to assign them meaning when we find them. Kahneman and Tversky analyzed many of the shortcuts we employ in assessing patterns in data and in making judgments in the face of uncertainty. They dubbed those shortcuts heuristics. In general, heuristics are useful, but just as our manner of processing optical information sometimes leads to optical illusions, so heuristics sometimes lead