The Filter Bubble - Eli Pariser [84]
While many of these data acquisitions will be legitimate, some won’t be. Data are uniquely suited to gray-market activities, because they need not carry any trace of where they have come from or where they have been along the way. Wright calls this data laundering, and it’s already well under way: Spyware and spam companies sell questionably derived data to middlemen, who then add it to the databases powering the marketing campaigns of major corporations.
Moreover, because the transformations applied to your data are often opaque, it’s not always clear exactly what decisions are being made on your behalf, by whom, or to what end. This matters plenty when we’re talking about information streams, but it matters even more when this power is infused into our sensory apparatus itself.
In 2000, Bill Joy, the Sun Microsystems cofounder, wrote a piece for Wired magazine titled “Why the Future Doesn’t Need Us.” “As society and the problems that face it become more and more complex and machines become more and more intelligent,” he wrote, “people will let machines make more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones.”
That may often be the case: Machine-driven systems do provide significant value. The whole promise of these technologies is that they give us more freedom and more control over our world—lights that respond to our whims and moods, screens and overlays that allow us to attend only to the people we want to, so that we don’t have to do the busywork of living. The irony is that they offer this freedom and control by taking it away. It’s one thing when a remote control’s array of buttons elides our ability to do something basic like flip the channels. It’s another thing when what the remote controls is our lives.
It’s fair to guess that the technology of the future will work about as well as the technology of the past—which is to say, well enough, but not perfectly. There will be bugs. There will be dislocations and annoyances. There will be breakdowns that cause us to question whether the whole system was worth it in the first place. And we’ll live with the threat that systems made to support us will be turned against us—that a clever hacker who cracks the baby monitor now has a surveillance device, that someone who can interfere with what we see can expose us to danger. The more power we have over our own environments, the more power someone who assumes the controls has over us.
That is why it’s worth keeping the basic logic of these systems in mind: You don’t get to create your world on your own. You live in an equilibrium between your own desires and what the market will bear. And while in many cases this provides for healthier, happier lives, it also provides for the commercialization of everything—even of our sensory apparatus itself. There are few things uglier to contemplate than AugCog-enabled ads that escalate until they seize control of your attention.
We’re compelled to return to Jaron Lanier’s question: For whom do these technologies work? If history is any guide, we may not be the primary customer. And as technology gets better and better at directing our attention, we need to watch closely what it is directing our attention toward.
8
Escape from the City of Ghettos
In order to find his own self, [a person] also needs to live in a milieu where the possibility of many different value systems is explicitly recognized and honored. More specifically, he needs a great variety of choices so that he is not misled about the nature of his own person.
—Christopher Alexander et al., A Pattern Language
In theory, there’s never been a structure more capable of allowing