Proofiness - Charles Seife [23]
Even though the lines seem very convincing at first glance, they’re not truly capturing the underlying pattern in the data. Women have been racing competitively for a shorter time than men have, so it’s no surprise that women’s race times have been improving very rapidly, at least compared to men. As the sport matures, the improvement will slow down; the line will shift course, flattening out. Eventually, as both men and women reach their physiological limits, the improvement will stop entirely. The lines will be horizontal.
Figure 10. A more likely scenario for future sprints.
The lines need never cross at all; indeed, it’s quite possible that the men’s and women’s sprint times will get closer for many years, but the women will never beat the men. What’s certain, though, is that the data can’t follow straight lines indefinitely. The laws of nature say so.
Mere natural law didn’t stop scientists from making their bogus prediction, grabbing headlines with the prophecy that women will outsprint men in 2156. And it didn’t stop Nature editors from accepting the paper. Nature should certainly have known better, and not just because of the inherent silliness of having women breaking the sound barrier in the middle of the millennium. In fact, the journal had been caught before with exactly the same sort of stupid prediction.
In 1992, two physiologists looked at women’s and men’s race times and blithely drew meaningless lines through the data. Their conclusion: female marathon runners would beat men after 1998, with a time of 2:01:59.00.24 Not even close. The female gold medalist in the 2000 Olympic Marathon had a time of 2:23:14, while her male counterpart outpaced her handily, beating her time by more than thirteen minutes. Despite the bold prediction from Nature—which was repeated credulously by the New York Times and other newspapers—female marathoners still lag behind their male counterparts by about fifteen minutes, and even the male world record is still a few minutes away from the 2:01:59 mark that should have been broken more than a decade ago.
It’s trivially easy to generate a line, curve, equation, or formula that seems to describe the pattern in a set of data yet has no real value at all. These faux patterns look convincing, dressed up as they are in mathematical language. But when people try to use them to predict something—to exploit the pattern to say something new about the universe—they fall completely flat. Nevertheless, scientists, economists, public health experts, and all kinds of people with access to basic statistical software crank out meaningless curves, lines, equations, and formulae at an incredible pace. This is yet another form of proofiness: regression to the moon.
Regression analysis is a mathematical tool that people use to create lines, curves, formulae, or equations that fit a set of data. It’s an extremely powerful technique; it quickly extracts a pattern from whatever data you provide it. However, when it is used incorrectly, the results are meaningless or downright barmy. The female-versus-male race times are obvious examples because the pattern that the scientists found in their data was inherently absurd; taken literally, it leads to the conclusion that runners will travel backward in time. Often the problems are more subtle.
In the 1980s, economists were ga-ga over a piece of research coming out of Yale University. A young economist, Ray Fair, had done a regression analysis on economic data from 1912 to 1976 and came up with an equation that, if correct, had stunning consequences—it predicted, well ahead of time, who would win a presidential election. All you had to do was plug in a few economic indicators—inflation, growth rate, and a few other factors—and voilà, out pops the next president. War, domestic issues, foreign policy—all of these were more or less irrelevant; the economic situation determines the winner of any given election. It’s an equation that economists were sure to love.
Fair’s equation predicted that Reagan would beat Carter with