• Choose a category
• All
• Classic-Fiction

By Root 289 0

had several researchers trolling the Internet for Jeopardy data. If this system was going to compete with humans in the game, it would require two types of information. First, it needed Jeopardy clues, thousands of them. This would be the machine’s study guide—what those in the field of machine learning called a training set. A human player might watch a few Jeopardy shows to get a feel for the types of clues and then take some time to study country capitals or brush up on Shakespeare. The computer would do the same work statistically. Each Jeopardy clue, of course, was unique and would never be repeated, so it wasn’t a question of learning the answers. But a training set would orient the researchers. Given thousands of clues, IBM programmers could see what percentage of them dealt with geography, U.S. presidents, words in a foreign language, soap operas, and hundreds of other categories—and how much detail the computer would need for each. The clue asking which presidential candidate carried New York State in 1948, for example (“Who is Thomas Dewey?”), indicated that the computer would have to keep track of White House losers as well as winners. What were the odds of a presidential loser popping up in a clue?

Digging through the training set, researchers could also rank various categories of puzzles and word games. They could calculate the odds that a Jeopardy match would include a puzzling Before & After, asking, for example, about the “Kill Bill star who played 11 seasons behind the plate for the New York Yankees” (“Who is Uma Thurman Munson?”). A rich training set would give them a chance to scrutinize the language in Jeopardy clues, including abbreviations, slang, and foreign words. If the machine didn’t recognize AKA as “also known as” or “oops!” as a misunderstanding, if it didn’t recognize “sayonara,” “au revoir,” “auf Wiedersehen,” and hundreds of other expressions, it could kiss entire Jeopardy categories goodbye. Without a good training set, researchers might be filling the brain of their bionic student with the wrong information.

Second, and nearly as important, they needed data on the performance of past Jeopardy champs. How often did they get the questions right? How long did they take to buzz in? What were their betting strategies in Double Jeopardy and Final Jeopardy? These humans were the competition, and their performance became the benchmark for Blue J.

In the end, it didn’t take a team of sleuths to track down much of this data. With a simple Internet search, they found a Web site called J! Archive, a trove of historical Jeopardy data. A labor of love by Jeopardy fans, the site detailed every game in the show’s history, with the clues, the contestants, their answers—and even the comments by Alex Trebek. Here were more than 180,000 clues, hundreds of categories, and the performance of thousands of players, from first-time losers to champions like Brad Rutter and Ken Jennings.

In these early days, the researchers focused only on Jennings. He was the gold standard. And with records of his seventy-four games—more than four times as many as any other champion—they could study his patterns, his strengths and vulnerabilities. They designed a chart, the Jennings Arc, to map his performance: the percentage of questions on which he won the buzz and his precision on those questions. Each of his games was represented by a dot, and the best ones, with high buzz and high accuracy, floated high on the chart to the extreme right. His precision averaged 92 percent and occasionally reached 100 percent. He routinely dominated the buzz, on one game answering an astounding 75 percent of the clues. For each of these games, the IBM team calculated how well a competitor would have to perform to beat him. The numbers varied, but it was clear that their machine would need to win the buzz at least half the time, get about nine of ten right—and also win its share of Daily Doubles.

In the early summer of 2007, after the bake-off, the Jeopardy team marked the performance of the Piquant system on the Jennings Arc. (Basement Baseline,