The Hidden Reality_ Parallel Universes and the Deep Laws of the Cosmos - Brian Greene [141]
To illustrate, imagine that Oscar has straightened up his room, except that the thousand silver dollars he won in last week’s poker game remain scattered across the floor. Even after he gathers them in a neat cluster, Oscar sees only a haphazard assortment of dollar coins, some heads and others tails. Were you to randomly change some heads to tails and other tails to heads, he’d never notice—evidence that the thousand-dropped-silver-dollar system has high entropy. Indeed, this example is so explicit that we can do the entropy counting. If there were only two coins, there’d be four possible configurations: (heads, heads), (heads, tails), (tails, heads), and (tails, tails)—two possibilities for the first dollar, times two for the second. With three coins, there’d be eight possible arrangements: (heads, heads, heads), (heads, heads, tails), (heads, tails, heads), (heads, tails, tails), (tails, heads, heads), (tails, heads, tails), (tails, tails, heads), (tails, tails, tails), arising from two possibilities for the first, times two for the second, times two for the third. With a thousand coins, the number of possibilities follows exactly the same pattern—a factor of 2 for each coin—yielding a total of 21000, which is . The vast majority of these heads-tails arrangements would have no distinguishing features, so they would not stand out in any way. Some would, for instance, if all 1,000 coins were heads or all were tails, or if 999 were heads, or 999 tails. But the number of such unusual configurations is so extraordinarily small, compared with the huge total number of possibilities, that removing them from the count would hardly make a difference.*
From our earlier discussion, you’d deduce that the number 21000 is the entropy of the coins. And, for some purposes, that conclusion would be fine. But to draw the strongest link between entropy and information, I need to sharpen up the description I gave earlier. The entropy of a system is related to the number of indistinguishable rearrangements of its constituents, but properly speaking is not equal to the number itself. The relationship is expressed by a mathematical operation called a logarithm; don’t be put off if this brings back bad memories of high school math class. In our coin example, it simply means that you pick out the exponent in the number of rearrangements—that is, the entropy is defined as 1,000 rather than 21000.
Using logarithms has the advantage of allowing us to work with more manageable numbers, but there’s a more important motivation. Imagine I ask you how much information you’d need to supply in order to describe one particular heads-tails arrangement of the 1,000 coins. The simplest response is that you’d need to provide the list—heads, heads, tails, heads, tails, tails …—that specifies the disposition of each of the 1,000 coins. Sure, I respond, that would tell me the details of the configuration, but that wasn’t my question. I asked how much information is contained in that list.
So, you start to ponder. What actually is information, and what does it do? Your response is simple and direct. Information answers questions. Years of research by mathematicians, physicists, and computer scientists have made this precise. Their investigations have established that the most useful measure of information content is the number of distinct yes-no questions the information can answer. The coins’ information answers 1,000 such questions: Is the first dollar heads? Yes. Is the second dollar heads? Yes. Is the third dollar heads? No. Is the fourth dollar heads? No. And so on. A datum that can answer a single yes-no question is called a bit—a familiar computer-age term that is short for binary digit, meaning a 0 or a 1, which you can think of as a numerical representation of yes