• Choose a category
• All
• Classic-Fiction

By Root 312 0

the data they produced, not the spectacle, and much less the laughs. As she saw it, the team had a long list of improvements to make before autumn. By that point, the immense collection of software running Watson would be locked down—frozen. After that, the only tinkering would be in a few peripheral applications, like game strategy. But the central operations of the computer, like those of other mission-critical systems, would go through little but testing during the months leading up to the Jeopardy showdown. Engineers didn’t dare tinker with Space Shuttle software once the vessel was headed toward the launch pad. Watson would get similar treatment.

With each sparring session, however, the list of fixes was getting longer. For each fix, the team had to weigh the time it would take against the possible gain in performance. “It’s triage,” Chu-Carroll said. During one sparring session, for example, Watson mispronounced weinerschnitzel, neglecting to say the “W” as a “V.” Was it worth the trouble to fine-tune its German phonetics? Not unless someone could do it in a hurry.

In one Final Jeopardy, Watson inched closer to the fix-it threshold. Asked to identify the sole character in the American Film Institute’s list of the fifty greatest heroes who was not portrayed by a human, the computer came back with “Who is Buffy the Vampire Slayer?” The audience laughed, and Todd Crain slapped his forehead, saying, “Oh, Watson, for the love of God!”

Still, solving that clue would have been a formidable challenge. Once Watson found the list of heroes, it would have had to carry out fifty separate searches to ascertain that each of the characters, from Atticus Finch to James Bond, Indiana Jones, and Casablanca’s Rick Blaine, was human. (It wouldn’t necessarily be that easy, since most documents and databases don’t note a protagonist’s species.) During that search, presumably, it would see that thirty-ninth on the list was a collie, a breed of dog (and therefore not human), and would then display “Who is Lassie?” on its electronic screen. Would the lessons gained in learning how to spot the dog in a long list of humans pay off elsewhere? Probably not.

That raised another question for the harried team. If Watson had abysmally low confidence in a Final Jeopardy response, as was the case with the Pet Shop Boys and Buffy the Vampire Slayer, would it be better to say nothing? If it was in the company’s interest to avoid looking stupid, suppressing wild guesses might be a good move. This dilemma did not arise with the regular Jeopardy clues. There, if Watson lacked confidence in an answer, it simply refrained from buzzing. But in Daily Doubles and Final Jeopardy, contestants had to bet before seeing the clue. Humans guessed when they didn’t know the answer. This is what Watson was doing, too. But its chronic shortage of common sense made its guesses infinitely dumber. In the coming weeks, the IBM team would calculate the odds of a lucky guess for each of Watson’s confidence levels. While Jeopardy executives, eager for entertainment and high ratings, would no doubt favor the occasional outrageous guess, IBM had other priorities. “At low levels of confidence, I think we’ll just have it say it doesn’t know,” said Chu-Carroll. “Sometimes that sounds smarter.”

Mathematics was one category where the IBM machine could not afford to look dumb. The company, after all, was built on math. However, the Jeopardy training data didn’t include enough examples to educate Watson in this area. Of the more than seventy-five thousand clues Eric Brown and his team studied, only fourteen involved operations with fractions. A game strategist wouldn’t dwell on them. But for IBM, there was more at risk than winning or losing a game. To prepare Watson for math, the team might have to put aside the statistical approach and train the machine in the rules and lingo of arithmetic.

As they worked to lift Watson’s performance, the Jeopardy team focused on entire categories that the machine misunderstood. They called them train wrecks. It was a new genre, conceived after