Proofiness - Charles Seife [96]
“Heads? Bush, then. Thank you very much for your time.”
It costs money to ask people questions, and your company is very cheap. They’ve only given you the cash to poll sixteen people in all. Flip a coin sixteen times and count up the number of heads and the number of tails.
Did your poll get the right answer? Did you show that the electorate was split right down the middle, with eight people voting for Bush and eight people voting for Gore? Probably not. There’s only about a one in five chance that your sixteen answers split exactly evenly, giving you the correct answer. Most of the time, your poll is slightly wrong. Often, you will get nine Bush votes to seven Gore votes, making it look as if Bush has a 56 percent to 44 percent lead over Gore. Similarly, it’s quite likely that you get seven Bush votes to nine Gore votes, making it appear that Gore has the equivalent lead. It’s also common for a pollster to get ten Bush votes to six Gore votes or vice versa—making it seem as if one of the candidates has a huge 62 percent to 38 percent lead.
If you did the exact same poll over and over (try it!), you’d see that the results jump around quite a bit. Sometimes your poll is dead accurate—it gives exactly the right answer, showing that the candidates are neck and neck. However, more often, through random chance, you happen to get an extra Bush or Gore supporter or two in your sample, artificially making it look as if one candidate has a lead over the other. Fully 80 percent of the time, you’re going to get the wrong answer—just because of dumb luck. This is statistical error, and it’s an unavoidable consequence of sampling. The laws of randomness inject some uncertainty into the polling process. You’re never certain that your answer represents the truth.
Luckily, statistical error is extremely predictable. If you did your poll of sixteen people over and over again and plotted the results, you would see that they followed a pattern—a bell curve (above).
Figure A1. Expected outcomes of polls of 16 people.
If there were no such thing as statistical error, if your poll result always was a perfect reflection of the truth, it wouldn’t be much of a curve at all; it would just be a pencil-thin line sitting on the 50 percent mark. Every single time, your poll would give an answer of 50 percent for Bush, 50 percent for Gore. However, that’s not the way the world works. Statistical error ensures that you’ve got a nice fat curve. Once in a while, your poll gives you the right answer, but most of the time, your results are off—often by quite a bit. The fatter the curve, the more likely it is that your poll is really far away from the truth.
There’s nothing you can do to eliminate statistical error, but you can make it smaller. If you can convince your corporate bosses to shell out a little more money and interview sixty-four people instead of sixteen, the curve gets somewhat thinner (above).
Figure A2. Expected outcomes of polls of 64 people.
This means that your poll is less likely to be wildly wrong. The more people you poll, the less frequently random errors will throw off your answer by a large amount. Those errors never completely disappear, but you can shrink them to a reasonable size by including lots and lots of people in your poll. If you managed to poll 1,024 people, for example, the curve would look like the one on page 249.
As you can tell, most of the time your poll results will be relatively close to the truth. Unlike your poll of sixteen people, where it was quite common to see results that showed Bush or Gore with a twenty-point lead, that almost never happens with a poll of 1,024 people.
What about margin of error? It’s essentially a measure of the fatness of the curve. The margins of error are the lines, marked on the preceding graphs, such that 95 percent of the bell curve falls between them. In the sixteen-person poll, roughly 95 percent of the time the answer you get will be somewhere