• Choose a category
• All
• Classic-Fiction

By Root 1058 0

It is simply harder to predict the box office potential of a movie at green light stage than a week or two before its release, no matter what methods you use. In the same way, predictions about new product sales, say, are likely to be less accurate than predictions about the sales of existing products no matter when you make them. There’s nothing you can do about that, but what you can do is start using any one of several different methods—or even use all of them together, as we did in our study of prediction markets—and keep track of their performance over time. As I mentioned at the beginning of the previous chapter, keeping track of our predictions is not something that comes naturally to us: We make lots of predictions, but rarely check back to see how often we got them right. But keeping track of performance is possibly the most important activity of all—because only then can you learn how accurately it is possible to predict, and therefore how much weight you should put on the predictions you make.12

FUTURE SHOCK

No matter how carefully you adhere to this advice, a serious limitation with all prediction methods is that they are only reliable to the extent that the same kind of events will happen in the future as happened in the past, and with the same average frequency.13 In regular times, for example, credit card companies may be able to do a pretty good job of predicting default rates. Individual people may be complicated and unpredictable, but they tend to be complicated and unpredictable in much the same way this week as they were last week, and so on average the models work reasonably well. But as many critics of predictive modeling have pointed out, many of the outcomes that we care about most—like the onset of the financial crisis, the emergence of a revolutionary new technology, the overthrow of an oppressive regime, or a precipitous drop in violent crime—are interesting to us precisely because they are not regular times. And in these situations some very serious problems arise from relying on historical data to predict future outcomes—as a number of credit card companies discovered when default rates soared in the aftermath of the recent financial crisis.

Even more important, the models that many banks were using to price mortgage-backed derivatives prior to 2008—like the infamous CDOs—now seem to have relied too much on data from the recent past, during which time housing prices had only gone up. As a result, ratings analysts and traders alike collectively placed too low a probability on a nationwide drop in real-estate values, and so badly underestimated the risk of mortgage defaults and foreclosure rates.14 At first, it might seem that this would have been a perfect application for prediction markets, which might have done a better job of anticipating the crisis than all the “quants” working in the banks. But in fact it would have been precisely these people—along with the politicians, government regulators, and other financial market specialists who also failed to anticipate the crisis—who would have been participating in the prediction market, so it’s unlikely that the wisdom of crowds would have been any help at all. Arguably, in fact, it was precisely the “wisdom” of the crowd that got us into the mess in the first place. So if models, markets, and crowds can’t help predict black swan events like the financial crisis, then what are we supposed to do about them?

A second problem with methods that rely on historical data is that big, strategic decisions are not made frequently enough to benefit from a statistical approach. It may be the case, historically speaking, that most wars end poorly, or that most corporate mergers don’t pay off. But it may also be true that some military interventions are justified and that some mergers succeed, and it may be impossible to tell the difference in advance. If you could make millions, or even hundreds, of such bets, it would make sense to go with the historical probabilities. But when facing a decision about whether or not to lead the country into war, or to