I'm Feeling Lucky_ The Confessions of Google Employee Number 59 - Douglas Edwards [108]
Googlers annotated the chart with their own hire dates and other events of significance (such as the founding of the Google wine club). Celebrity-guest signatures appeared with regularity. Al Gore was there. Jimmy Carter. Chris Martin from Coldplay and his girlfriend Gwyneth Paltrow. Claus quickly decreased the scale of the graph so that three hundred thousand searches no longer appeared as a hill reaching halfway up the page. That mark now represented three million searches. Within months, he recalibrated so that the same height equaled thirty million searches. And still the mountains grew until they scraped the top of the chart.
Schwim sent out emails announcing major increments in the number of searches per day on Google.com, which started at around four million when I was hired in late 1999. Less than a month later that number, not including partners, was more than five million. Two months later we crossed eight million, and by the middle of September 2000 we were conducting a million searches an hour. Ninety days after that, it was a thousand searches per second. Claus added taller pieces of paper to the side of the graph as the volume of our traffic marched inexorably upward and to the right.
Operations had shifted into high gear on machine construction for the Yahoo deal, and now the production pedal was nailed to the floor. Fleets of computers provided enough capacity that a whole data center could be taken offline without bringing down Google or its partners. When a new index was built, the techs copied all the files to one data center and then moved on to update the next one. That caused some weirdness in our results.
"For about a week each month," Matt Cutts recalls, "depending on randomness and chance, you'd either go to the old data center or a new data center." If your search was routed to a data center that hadn't been updated and then you did the same search again at one that had, your results would appear to jump around, as if they were dancing. The "Google Dance" became an event of great concern to those who cared about their ranking in search results. On the WebmasterWorld website, commenters named each index switchover as if it were a hurricane, and the outcome could be equally devastating for any business whose website dropped off the first page of results. For Google, the switchover was not just about adding more data to the index. It was a time to tweak the ranking algorithms as well.
Google's focus moved back and forth from search quality to infrastructure. The initial innovation of PageRank at Stanford improved search relevance. The systems work in early 2000 led to the billion-URL index and faster response rates. In 2001, the pendulum swung back to search quality.
"Larry used to say, 'Search is too cheap,'" Urs told me. "The cost per query is really too low. Given the revenue we have, we should be able to do much more." Spending money to improve search quality would create a perceptible gap between Google's results and those of our competitors, enabling us to build a brand based on quality. Other search companies would have to invest at an equal or greater rate just to catch up. We would launch an arms race and spend our opponents into bankruptcy.
"Raising the cost of search will in fact increase the profit," Urs believed. "Your top line is going to go up much faster than your bottom line, and your margin is really very large."
Larry and Sergey wanted a focus on efficiency, but they also wanted to find out how much better Google could get if we threw twice as many machines at the problem of search quality. So they tried an experiment.
"In 2001," Sanjay recalls, "I was sitting in Larry's office and he just said, 'Here are three thousand machines. You guys figure out something to do with that.' It wasn't like we just happened to have all these spare