IBM Puts Supercomputer In 'Jeopardy!'

(Inside Science) -- “Let’s finish, ‘Chicks Dig Me’,” intones the somewhat monotone, but not unpleasant, voice of Watson, IBM’s new supercomputer built to compete on the game show Jeopardy! 

The audience chuckles in response to the machine-like voice and its all-too-human assertion. But fellow contestant Ken Jennings gets the last laugh as he buzzes in and garners $1,000. 

This exchange is part of a January 13 practice round for the world’s first man vs. machine game show. Scheduled to air February 14-16, the match pits Watson against the two best Jeopardy! players of all time. Jennings holds the record for the most consecutive games won, at 74. The other contestant, Brad Rutter, has winnings totaling over $3.2 million. 

At the contestants’ podium, Watson appears as an atom-like icon sporting the splayed lines of an idea lightbulb. But behind its monitor is some of the most sophisticated computer science ever assembled. A core team of 25 IBM programmers spent four years building Watson, the world’s most advanced question and answer system.

Responding to Jeopardy! questions is tougher than the kind of search that Google does because it requires a single answer, not pages of possibilities. Confounding the problem, said Raymond Mooney, a computer science professor at The University of Texas at Austin, is that the kind of questions that make Jeopardy! interesting are riddled with ambiguity, allusions and puns.

David Ferrucci, program manager of the Watson project, explained that language comes naturally to people, because we think in words and phrases. But computers, at their core zeros and ones, process information mathematically. And there’s no formal mathematics for everyday language.

When Watson receives input, it parses it against a vast database including encyclopedias, textbooks and news archives, along with sources favored by Jeopardy! writers like the complete works of Shakespeare and the Bible. 

Watson finds several-hundred candidate answers, which are scored by thousands of algorithms. It then compares the highest score against a threshold. The threshold changes depending on how much risk Watson will accept. “One way to think about it is for every question and answer we have lots of pieces of evidence, then we score each to come up with a single answer,” explained David Gondek, a member of Watson’s algorithms and strategy teams.

Watson runs on 3,000 cores simultaneously, an example of the aptly named “massively parallel computing.” Such hefty computational rigging reveals its most preferred answer in seconds. A typical PC requires about two hours for the same process.

This fall Watson played 55 sparring matches against Tournament of Champions winners. The training taught Watson that some algorithms are more trustworthy than others. The computer awards these algorithms higher scores. 

Mooney, who attended several sparring matches but did not contribute to the Watson project, said he was skeptical that a machine could compete on Jeopardy! but became convinced after seeing Watson in action. “They’ve done a great job of putting together a lot of existing artificial intelligence technology into a very complicated system that actually works on this problem.” 

Watson receives its input in text form -- it doesn’t use speech recognition software -- at the same moment the contestants see it. Host Alex Trebek then reads the question out loud. When he’s done speaking, a light signals that the buzzers are open. Watson receives an electronic signal at the same moment and will buzz in only if it has an answer with a high enough score in time. 

Part of the fun of watching Watson is the answer panel that shows viewers the machine’s top three candidate answers and their scores, along with the threshold, which is lower when Watson trails in a game and increases when it’s ahead.

“It’s something that the team is watching closely during the game, too,“ said Gondek.

If the January 13 match is any indication, Watson has both the speed and accuracy to put up a fight. While neither man nor machine missed a single question during the 5-minute parry, Watson finished the round with $4,400, ahead of Jennings with $3,400 and Rutter with $1,200. 

A representative from IBM reported that no changes were made to Watson following the January 13 round, “The system is now locked and has not been refined since the practice match.”

As of this writing, the all-human poll on the IBM website gives a high score to Watson: 53 percent of the respondents think the next Jeopardy! Champion will be a machine.