Final Jeopardy (Alexandra Cooper Mysteries) - Linda Fairstein [59]
But why didn’t Watson find the right answer? For a computer with access to millions of documents and lists, the July 1, 1946, cover profile in the nation’s leading newsmagazine shouldn’t be a deep mystery.
Ferrucci concluded that something was wrong with Watson and he wanted the team in the War Room at Hawthorne to get working on it right away. Yet even in one of the world’s leading technology companies, it wasn’t clear how to send the digital record of the computer’s misadventures through the Internet. Ferrucci asked Eric Brown, then Eddie Epstein, and then Brown again: “How do I get the xml file to Hawthorne?” For Ferrucci, this failed game was brimming with vital feedback. It could point the Hawthorne team toward crucial fixes. The idea that his team could not respond immediately to whatever ailed Watson filled him with dread. Just imagine if Watson reprised this disastrous performance in its nationwide debut with Jennings and Rutter. “HOW DO I GET THIS FILE TO HAWTHORNE?” he shouted. No one had a quick answer. Ferrucci continued to thunder while, on the other side of the window, Todd Crain, Watson, and the other Jeopardy players blithely continued their game. (Watson, for one, was completely unfazed.) Finally Brown confirmed that he could plug a thumb drive into one of Watson’s boxes, download the game data, and e-mail it to the team in Hawthorne. It promised to be a long night in the War Room, as the researchers diagnosed Watson’s flops and struggled to restore its cognitive mojo.
Cloistered in a refrigerated room on the third floor of the Hawthorne labs stood another version of Watson. It turned out that the team needed two Watsons: the game player, engineered for speed, and this slower, steadier, and more forgiving system for development. The speedy Watson, its algorithms deployed across more than 2,000 processors, was a finicky beast and near impossible to tinker with. This slower Watson kept running while developers rewrote certain instructions, shifted out one algorithm for another, or refined its betting strategy. It took forty minutes to run a batch of questions, but it could handle two hundred at a time. Unlike the fast machine, it created meticulous records, and it permitted researchers to experiment, section by section, with its answering process. Because the team could fiddle with the slower machine, it was always up-to-date, usually a month or two ahead of its speedy sibling. After the debacle against Lindsay, IBM could only hope that the slower, smarter Watson wouldn’t have been so confused.
Within twenty-four hours, Ferrucci’s team had run all of that day’s games on the slow machine. The news was encouraging. It performed 10 percent better on the clues. The biggest difference, according to Eric Brown, was that some of the clues were topical, and speedy Watson’s most recent data came from 2008. “We got creamed on a couple categories that required much more current information,” he said.
Other recent adjustments in the slow Watson helped it deal with chronology. Keeping track of facts as they change over time is a chronic problem for AI systems, and Watson was no exception. In the recent sparring session, it had confused a mid-nineteenth-century novel for a late-twentieth-century pop duo. Yet when Ferrucci analyzed the slower Watson’s performance on the problematic Oliver Twist clue, he was relieved to see that a recent tweak had helped the machine match the clue to the right century. This fix in “temporal reasoning” pushed the Pet Shop Boys answer way down its list, from first to number 79. Watson’s latest top answer—“What is magician?”—was still wrong but not as laughable. “It still knows nothing about Oliver Twist,” Ferrucci wrote in a late-night e-mail.
While Ferrucci and a handful of team members attended every sparring match in the winter of 2010, Jennifer Chu-Carroll generally stayed away. For her, their value was in