Knocking on Heaven's Door - Lisa Randall [101]
It was interesting to talk to Nate about the kind of things he tries to predict. A lot of recent popular books present shaky hypotheses that give predictions that work—except when they don’t. Nate is a lot more scientific. He first became famous for his accurate predictions of baseball games and elections. His analysis was based on careful statistical evaluations of similar situations in the past, where he included as many variables that he could manage to apply historical lessons to as precisely as he could.
He now has to choose wisely where to apply his methods. But he realizes that the kinds of correlations he focuses on can be tricky to interpret. You can say an engine on fire caused a plane crash, but it’s not a surprise to find an engine on fire in a plane going down. What really was the initial cause? You have the same issue when you connect a mutated gene to cancer. It doesn’t necessarily cause the disease even if it is correlated with it.
He is aware of other potential pifalls too. Even with large amounts of data, randomness and noise may enhance or suppress the interesting underlying signals. So Nate won’t work on financial markets or earthquakes or climate. Although in all likelihood he could predict overall trends, the short-term predictions would be inherently uncertain. Nate now studies other places where his methods shed light such as how best to distribute music and movies, as well as questions such as the value of NBA superstars. But he acknowledges that only very few systems can be so accurately quantified.
Nonetheless, Nate told me that forecasters do make one other type of prediction. Many of them do metaforecasting—predicting what people will try to predict.
CHAPTER TWELVE
MEASUREMENT AND UNCERTAINTY
Familiarity and comfort with statistics and probability help when evaluating scientific measurements, not to mention many of the difficult issues of today’s complex world. I was reminded of the virtue of probabilistic reasoning when, a few years back, a friend was frustrated by my “I don’t know” response to his question about whether or not I planned to attend an event the following evening. Fortunately for me, he was a gambler and mathematically inclined. So instead of exasperatingly insisting on a definite reply, he asked me to tell him the odds. To my surprise, I found that question a lot simpler to deal with. Even though the probability estimate I gave him was only a rough guess, it more closely reflected my competing considerations and uncertainties than a definite yes or no reply would have done. In the end, it felt like a more honest response.
Since then I’ve tried this probabilistic approach out on friends and colleagues when they didn’t think they could reply to a question. I’ve found that most people—scientists or not—have strong but not irrevocable opinions that they frequently feel more comfortable expressing probabilistically. Someone might not know if he wants to go to the baseball game on the Thursday three weeks from now. But if he knows that he likes baseball and doesn’t think he has any work trips coming up, yet hesitates because it’s during the week, he might agree he is 80 percent likely to, even if he can’t give a definite yes. Although just an estimate, this probability—even one he makes up on the spot—more accurately reflects his true expectation.
In our conversation about science and how scientists operate, the screenwriter and director Mark Vicente observed how he was struck by the way that scientists hesitate to make definite unqualified