I'm Feeling Lucky_ The Confessions of Google Employee Number 59 - Douglas Edwards [12]
I stopped by the office Schwim shared with Jim so he could sign me up for a company email account. A large stuffed penguin, the mascot for the Linux operating system our engineers used instead of Windows, sat in a folding chair next to a model of the human skull left over from Jim's med school days. The room felt cramped, as most Google offices did, and was crowded with wires and RAM and computers in various states of assembly. Schwim peered from behind his monitor with the distracted look of someone whose mind was elsewhere—like John Malkovich in that movie where a puppeteer took command of his brain.
"You're the first Doug," Schwim told me. "Do you want doug@google.com?" I did. I felt a strange tingle as I thought about the implication of that. The first Doug. Among certain sets in Silicon Valley, your email address indicates more about you than the car you drive or the clothes you wear. I liked the status doug@google conferred on me as an early adopter.
I'd see a lot of "Jim and Schwim," as they came to be called. Their group, known as operations or ops, took charge of building and maintaining all the machines running Google. Larry had given Jim a list on his second day, in priority order, of the top one hundred things he wanted done. Number one was "to make sure we had enough capacity to run the site and if there are problems, solve them or find someone to solve them."
"The first year I got nine done," Jim confessed with a hint of pride. "And in the subsequent five years, I got through fifteen of them." Jim's job wasn't defined by the list, however, just as I was about to learn that mine wasn't defined by my somewhat generic "marketing manager" title. As Jim pointed out, "When there were problems to be solved, whoever could solve them did, regardless of what their official title was."
CableFest '99
"We've got some work to do at our data center on Saturday," Cindy informed all of us in the marketing group toward the end of my first week on the job. "Bring warm clothes, because I understand it can get a bit chilly in there." It was our formal invitation to "volunteer" at Google's CableFest '99.
I was no expert on computer hardware. I had read an article or two about servers, hubs, and routers, but I pronounced "router" as if it rhymed with "tooter" instead of "outer." Given my profound lack of technical expertise and my bad computer karma, why would any company allow me in the same room as its computational nerve center? That requires a bit of explanation.
In late 1999, Google began accelerating its climb to market domination. The media started whispering about the first search engine that actually worked, and users began telling their friends to give Google a try. More users meant more queries, and that meant more machines to respond to them. Jim and Schwim worked balls-to-the-wall to add capacity. Unfortunately, computers had suddenly become very hard to get. At the height of the dot-com madness, suppliers were so busy with big customers that they couldn't be bothered fending off the hellhounds of demand snapping at Google's heels. A global shortage of RAM (memory) made it worse, and Google's system, which had never been all that robust, started wheezing asthmatically.
Part of the problem was that Google had built its system to fail.
"Build machines so cheap that we don't care if they fail. And if they fail, just ignore them until we get around to fixing them." That was Google's strategy, according to hardware designer Will Whitted, who joined the company in 2001. "That concept of using commodity parts and of being extremely fault tolerant, of writing the software in a way that the hardware didn't have to be very good, was just brilliant." But only if you could get the parts to fix the broken computers and keep adding new machines. Or if you could improve the machines' efficiency so you