Proofiness - Charles Seife [99]
Appendix C: The Prosecutor’s Fallacy
The prosecutor’s fallacy is a slimy little trick born out of the difficulty we humans have in interpreting probabilities. By failing to put a probability in the proper context, a lawyer can make it appear incontrovertible that a person is guilty when the case is far from certain.
To illustrate the fallacy, let’s move away from the court system for a moment. Imagine that researchers have just discovered a rare and deadly disease—Head-Exploding Syndrome (HES). It’s 100 percent fatal; if you contract it, you’re going to die a horrible, painful, and very messy death. Luckily, researchers have developed an extremely good blood test to tell whether you’ve got HES. It’s exquisitely accurate—there’s only a one in a million chance that the blood test gives the wrong answer. Using this test, doctors start screening the population to find HES cases.
So you go in to the doctor’s office, and the doctor draws your blood and leaves the room. A few minutes later, she returns, pale as a ghost. “The test came back positive,” she says. Since the test is so accurate—the chances of an error are one in a million—it’s virtually certain that you’ve got Head-Exploding Syndrome. There’s only a one in a million chance that the test is wrong. This means that there’s only a one in a million chance that you don’t have the disease . . . right?
Not so fast. There’s a piece of information that’s missing before you can conclude that you’ve really got HES. You need to know just how rare the disease is. As it happens, HES is extraordinarily rare—epidemiologists estimate that it afflicts one in a billion people around the world. That means that of the seven billion people on earth, we expect only seven of them to have the disease. This little bit of information is crucial, because it allows you to put your positive test in context.
The one in a million chance of an error in the test seems pretty small, but if you’re screening seven billion people, the one in a million chance of error means that roughly seven thousand patients are going to get a test result that gives the wrong answer. That is, seven thousand people around the planet will test positive on the test even though they don’t have the disease. Since only seven people in the entire world actually have HES, this means that the vast majority of people who get a positive test don’t actually have HES. Indeed, if you test positive, the probability that you have HES is 7 divided by 7,000—or 1 in 1,000. The chances are 999 in 1,000 that the test is wrong and you don’t have the disease. You can rest easy. The one in a million probability of an incorrect test was deceptive; the probability didn’t mean anything on its own. Only when you put that one in a million in context—comparing it to the one in a billion incidence of the disease—can you calculate your chance of actually having HES.
The lesson here is that you must always put probabilities in their proper context. It’s a fallacy to look at the chance of the test’s being wrong and equate that with the probability that you have a disease. Instead, you must compare the probability to the chances of having the disease in the first place—and when the disease is rare, it can make even a tiny probability of an error on a test loom large.
This is the prosecutor’s fallacy in a different form. Instead of blood tests and disease, the fallacy deals with evidence and guilt, but the mathematics is exactly the same. A lawyer presents a very small probability without putting it in the proper context. As a result, the small probability convinces the jury that the statement must be true. Had