Everything Is Obvious_ _Once You Know the Answer - Duncan J. Watts [68]
This relevance problem is fundamental, and can’t be eliminated simply by having more information or a smarter algorithm. For example, in his book about prediction the political scientist and “predictioneer” Bruce Bueno de Mesquita extols the power of game theory to predict the outcomes of complex political negotiations.14 Given the intrinsic unpredictability of complex systems, it seems unlikely that his computer models can in fact predict what he says they can. But leaving that aside for the moment, let’s look at the larger question of what they could predict even if they worked perfectly. Take for example his claim to have successfully predicted the outcome of the 1993 Oslo Accords between Israel and the then Palestine Liberation Organization. At the time, that would have seemed like an impressive feat. But what the algorithm didn’t predict was that the Oslo Accords were, in effect, a mirage, a temporary flicker of hope that was quickly extinguished by subsequent events. From what we now know about what happened afterward, in other words, it is clear that the outcome of the Oslo negotiations wasn’t the most important outcome to have predicted in the first place.
Of course, Bueno de Mesquita might reasonably point out that his models aren’t designed to make that sort of prediction. But that’s precisely the point: Making the right prediction is just as important as getting the prediction right. When we look back to the past, we do not wish that we had predicted what the search market share for Google would be in 1999, or how many days it would take for US soldiers to reach Baghdad during the second Gulf war. Those are certainly valid predictions that we might have thought to make. But at some point we would have realized that it didn’t really matter whether they were right or wrong—because they just weren’t that important. Instead we would end up wishing we’d been able to predict on the day of Google’s IPO that within a few years its stock price would peak above $500, because then we could have invested in it and become rich. We wish we’d been able to foresee the carnage that would follow the toppling of Saddam Hussein and the dismantling of his security forces, because then we could have adopted a different strategy or even avoided the whole mess in the first place.
Even when we are dealing with more mundane types of predictions—like how consumers will respond to such and such a color or design, or whether doctors would spend more time on preventative care if they were compensated on the basis of patients’ health outcomes rather than the number and expense of their prescribed procedures—we have the same problem. These sorts of predictions seem less problematic than predictions about the next great company or the next war. But as soon as we think about why we care about these predictions, we are forced immediately to make other predictions—about the effects of the predictions we’re making now. For example, we are concerned about how customers will react to the color not because we care about the reaction per se, but because we want the product to be a success, and we think the color will matter. Likewise, we care about the reaction of doctors to incentives because we wish to control healthcare costs and ultimately design a system that provides affordable healthcare to everyone without bankrupting