Chaos - James Gleick [43]
His description worked by making deeper and deeper separations between periods of clean transmission and periods of errors. Suppose you divided a day into hours. An hour might pass with no errors at all. Then an hour might contain errors. Then an hour might pass with no errors.
But suppose you then divided the hour with errors into smaller periods of twenty minutes. You would find that here, too, some periods would be completely clean, while some would contain a burst of errors. In fact, Mandelbrot argued—contrary to intuition—that you could never find a time during which errors were scattered continuously. Within any burst of errors, no matter how short, there would always be periods of completely error-free transmission. Furthermore, he discovered a consistent geometric relationship between the bursts of errors and the spaces of clean transmission. On scales of an hour or a second, the proportion of error-free periods to error-ridden periods remained constant. (Once, to Mandelbrot’s horror, a batch of data seemed to contradict his scheme—but it turned out that the engineers had failed to record the most extreme cases, on the assumption that they were irrelevant.)
Engineers had no framework for understanding Mandelbrot’s description, but mathematicians did. In effect, Mandelbrot was duplicating an abstract construction known as the Cantor set, after the nineteenth-century mathematician Georg Cantor. To make a Cantor set, you start with the interval of numbers from zero to one, represented by a line segment. Then you remove the middle third. That leaves two segments, and you remove the middle third of each (from one-ninth to two-ninths and from seven-ninths to eight-ninths). That leaves four segments, and you remove the middle third of each—and so on to infinity. What remains? A strange “dust” of points, arranged in clusters, infinitely many yet infinitely sparse. Mandelbrot was thinking of transmission errors as a Cantor set arranged in time.
This highly abstract description had practical weight for scientists trying to decide between different strategies of controlling error. In particular, it meant that, instead of trying to increase signal strength to drown out more and more noise, engineers should settle for a modest signal, accept the inevitability of errors and use a strategy of redundancy to catch and correct them. Mandelbrot also changed the way IBM’s engineers thought about the cause of noise. Bursts of errors had always sent the engineers looking for a man sticking a screwdriver somewhere. But Mandelbrot’s scaling patterns suggested that the noise would never be explained on the basis of specific local events.
Mandelbrot turned to other data, drawn from the world’s rivers. Egyptians have kept records of the height of the Nile for millennia. It is a matter of more than passing concern. The Nile suffers unusually great variation, flooding heavily in some years and subsiding in others. Mandelbrot classified the variation in terms of two kinds of effects, common in economics as well, which he called the Noah and Joseph Effects.
The Noah Effect means discontinuity: when a quantity changes, it can change almost arbitrarily fast. Economists traditionally imagined that prices change smoothly—rapidly or slowly, as the case may be, but smoothly in the sense that they pass through all the intervening levels on their way from one point to another. That image of motion was borrowed from physics, like much of the mathematics applied to economics. But it was wrong. Prices can change in instantaneous jumps, as swiftly as a piece of news can flash across a teletype wire and a thousand brokers can change their minds. A stock market strategy was doomed to fail, Mandelbrot argued, if it assumed that a stock would have to sell for $50 at some point on its way down from $60 to $10.
THE CANTOR DUST. Begin with a line; remove the middle third; then