I'm Feeling Lucky_ The Confessions of Google Employee Number 59 - Douglas Edwards [185]
The query scroll was carefully filtered for offensive terms that might clash with our wholesome image.* Offensive terms written in English, anyway. I recall a group of Japanese visitors pointing and smirking at some of the katakana characters floating across the page. The inability to identify foreign-language porn is just one of the reasons we never used the query scroll widely for marketing purposes, despite its ability to instantly turn esoteric technology into voyeuristic entertainment.
Larry never cared for the scrolling queries screen. He constantly monitored the currents of public paranoia around information seepage, and the scrolling queries set off his alarm. He felt the display could inadvertently reveal personal data, because queries could contain names or information that users would prefer to remain private (for example, "John Smith DUI arrest in Springfield" or "Mary Jones amateur porn movie"). Moreover, it might cause people to think more about their own queries and stir what he deemed to be ungrounded fears over what information was conveyed with each search.
Larry tried to kill the Google Zeitgeist, too. Zeitgeist was a year-end feature that the PR team put together recapping the trends in search terms over the previous twelve months. The press loved Zeitgeist because it gave them another way to wrap up the year, but to Larry it raised too many questions about how much Google knew about users' searches and how long we kept their data. Cindy asked me to come up with a list of reasons to continue the tradition, and my rationale evidently convinced Larry the risk was acceptable, because the year-end Zeitgeist is still published on Google.com.
All the while we wrestled with the issues of what to tell users, our ability to mine their data became better and better. Amit Patel, as his first big project at Google, had built a rudimentary system to make sense of the logs that recorded user interactions with our site. Ironically, the same engineer who did the most to seed the notion of "Don't be evil" in the company's consciousness also laid the cornerstone of a system that would bring into question the purity of Google's intentions.
Amit's system was a stopgap measure. It took three years and an enormous effort from a team of Googlers led by legendary coder Rob Pike to perfect the technology that, since it processed logs, came to be designated "Sawmill." The power of Sawmill when it was activated in 2003 gave Google a clear understanding of user behavior, which in turn enabled our engineers to serve ads more effectively than Yahoo did, to identify and block some types of robotic software submitting search terms, to report revenue accurately enough to meet audit requirements, and to determine which UI features improved the site and which confused users. If engineers were reluctant to delete logs data before Sawmill, they were adamant about retaining it afterward.
Larry's refusal to engage the privacy discussion with the public always frustrated me. I remained convinced we could start with basic information and build an information center that would be clear and forthright about the tradeoffs users made when they entered their queries on Google or any other search engine. I didn't really believe many people would read all the pages, or particularly care what they said. In fact, I somewhat cynically counted on that. The mere fact that we had the explanation available would allay many of their concerns. Those who truly cared would see we were being transparent. Even if they didn't like our policies on data collection or retention, they would know what they were. If they went elsewhere to search, they would be taking a chance that our competitors' practices were far worse than ours.
To Larry the risks were