The Filter Bubble - Eli Pariser [32]
For Heuer, the core lesson of the Nosenko debacle was clear: “Intelligence analysts should be self-conscious about their reasoning processes. They should think about how they make judgments and reach conclusions, not just about the judgments and conclusions themselves.”
Despite evidence to the contrary, Heuer wrote, we have a tendency to believe that the world is as it appears to be. Children eventually learn that a snack removed from view doesn’t disappear from the universe, but even as we mature we still tend to conflate seeing with believing. Philosophers call this view naïve realism, and it is as seductive as it is dangerous. We tend to believe we have full command of the facts and that the patterns we see in them are facts as well. (Angleton, the “master theory” proponent, was sure that Nosenko’s pattern of factual errors indicated that he was hiding something and was breaking under pressure.)
So what’s an intelligence analyst—or anyone who wants to get a good picture of the world, for that matter—to do? First, Heuer suggests, we have to realize that our idea of what’s real often comes to us secondhand and in a distorted form—edited, manipulated, and filtered through media, other human beings, and the many distorting elements of the human mind.
Nosenko’s case was riddled with these distorting factors, and the unreliability of the primary source was only the most obvious one. As voluminous as the set of data that the CIA had compiled on Nosenko was, it was incomplete in certain important ways: The agency knew a lot about his rank and status but had learned very little about his personal background and internal life. This led to a basic unquestioned assumption: “The KGB would never let a screw-up serve at this high level; therefore, he must be deceiving us.”
“To achieve the clearest possible image” of the world, Heuer writes, “analysts need more than information.... They also need to understand the lenses through which this information passes.” Some of these distorting lenses are outside of our heads. Like a biased sample in an experiment, a lopsided selection of data can create the wrong impression: For a number of structural and historical reasons, the CIA record on Nosenko was woefully inadequate when it came to the man’s personal history. And some of them are cognitive processes: We tend to convert “lots of pages of data” into “likely to be true,” for example. When several of them are at work at the same time, it becomes quite difficult to see what’s actually going on—a funhouse mirror reflecting a funhouse mirror reflecting reality.
This distorting effect is one of the challenges posed by personalized filters. Like a lens, the filter bubble invisibly transforms the world we experience by controlling what we see and don’t see. It interferes with the interplay between our mental processes and our external environment. In some ways, it can act like a magnifying glass, helpfully expanding our view of a niche area of knowledge. But at the same time, personalized filters limit what we are exposed to and therefore affect the way we think and learn. They can upset the delicate cognitive balance that helps us make good decisions and come up with new ideas. And because creativity is also a result of this interplay between mind and environment, they can get in the way of innovation. If we want to know what the world really looks like, we have to understand how filters shape and skew our view of it.
A Fine Balance