Watson

This is amazing. If I did not see it, I would have said it was impossible. Browse IBM’s pages for more.

Beating Ken Jennings at Jeopardy is simply an incredible achievement, far beyond beating Kasparov at chess.

The full games will air Feb 14th – 16th. I hope they made it sing “Daisy Bell” if it loses.

Update

I have been watching IBM’s YouTube videos, and I find the user comments depressing. Very few people are grasping what they have done here. I am trying to figure out how I would explain this to a non-technical family member. I think it is hard to project my feelings to someone who has never tried to program a machine to act “smart”.

Most people — including technology journalists — do not really know how computers work. Computers, like most technology, are essentially “magic”, and this all looks like just another example of the same kind of magic.

But I have a computer science degree from MIT. I know how your desktop works, and your iPhone, and your iPad, from the transistors all the way through the operating system and the software stack. I believe I could create Windows, or Linux, or Facebook, or Twitter, or even Google, given enough time. Put another way, if you transported me back 100 years, I could show them how to create all of these things… Or at least accelerate the process by a few decades.

But I could not even begin to tell you how IBM did this. Like I said, had I not seen it with my own eyes, I would have said it was impossible.

From where I sit, Watson is magic. I feel like I am watching a major historical event that will be lucky to register as a blip on the 24-hour news cycle.

10 comments to Watson

  • tplambeck

    I had the same initial reaction, but the more I’ve thought about it, the easier it seems.

    Think of enumerating all the possible Jeopardy answers. Who is X. What is Y, etc, for many datasets of X’s and Y’s. Big associative arrays, even PageRank-style algorithms just based on text in books, Almanacs, movie databases, whatever. From this huge database of possible answers, score each one in real time and pick the best possible answer.

    “During WWII, he led the Philippine…” Who is McArthur. (WWII, Philippine Islands, McArthur). Many millions of those things.

    The amazing thing is perhaps not that it’s possible, but how similar it looks to “human” information processing.

    This incipient Unheimlichkeit is only going to get eerier in the next couple of decades, as most of run-of-the-mill human discourse is revealed to be well-approximated by so many parrots chattering according to ever-more easily whipped up models that can respond in real time to things that people say.

    Nice blog…got here by following some muni threads

    Thane Plambeck

  • Kmicic

    Hello Nemo,
    Since Linux is free software, I have a simple question:
    How many patches have You commited to kernel tree, and how many have been accepted so far ??

  • Nemo

    tplambeck —

    Well, maybe our brains are nothing more than “so many parrots chattering”. But I still think that creating Watson is a lot harder than you estimate. Most AI problems will leave you thinking “come on, how hard could that be?” until you actually try to implement it. Human language in particular is extremely subtle. Dealing effectively with natural language in an essentially unrestricted context is more than any system has done to date, and more than I expected to see in my lifetime.

    Kmicic —

    I have contributed more than one patch to Linux, all but one of which have been accepted. I would give you details, including link(s) to git.kernel.org, but that would reveal my other name which is not going to happen. Decide for yourself whether I am full of crap.

  • tplambeck

    I agree that natural language understanding is difficult, but I suspect this Jeopardy thing can be done with little higher “interpretation” of Jeopardy questions. Instead, first build up a huge corpus of Jeopardy answers based on historical questions: “What is trigonometry”. “Who was Jean Paul Sartre.” “What is a black hole.” I think this is a lot of work, but it’s reasonably clear what the task is from an implementation standpoint. In the original posting, you seemed to be saying you had no idea how to proceed; I’m just saying it’s a lot of work, but reasonably clear how to proceed, at least with this first step.

    Then, take all the words in the question and consider them separately, for example, “Physicists believe one may be located at the center of our galaxy.” Or, (amazingly), as I have just done, just send them off ensemble to Google. Boom, I get the black hole Wikipedia entry. That’s way too easy and I just got lucky, but with better, Bayesian-informed models that score all words in the question separately, the task looks even more doable.

    BTW noticed you mentioned Martin Gardner. I’m on the board of the Gathering for Gardner. Maybe you’d like to come to one of our events sometime. tplambeck@gmail.com

  • tplambeck

    I’m also curious if Watson is required to “listen” to the question rather than have it simply served up as text. I’d view that as cheating on the task.

  • Nemo

    Instead, first build up a huge corpus of Jeopardy answers based on historical questions: “What is trigonometry”. “Who was Jean Paul Sartre.” “What is a black hole.”

    In one of the videos, one of the IBM researchers describes this approach. Then he immediately says this is not what they did because it would only give them a tiny fraction of a percent of the coverage needed. (Unfortunately I do not remember which video…)

    As for “listening” to the question… No, they receive it as text. I do not view this as cheating at all, since speech recognition is a largely independent problem that is also slowly being solved.

    What I really want to see are implementation details. I wonder how much they intend to release.

    I am also curious to see where this technology shows up in the next few years. It could probably replace level 1 tech support at Dell, Comcast, etc. And that’s just for starters.

  • Kmicic

    @ Nemo
    I don’t claim that You are full of crap. Infact, I do read your stuff.

  • alexrs

    I also have a computer science degree from MIT, and I’m only somewhat impressed. One of my friends (a grad student in AI) built a program in his spare time that can answer NYTimes crossword clues with reasonable accuracy. (Better than half of the time the real answer is in his program’s top 5 results, if I recall correctly.) He did this by processing old crossword clues and (more importantly) WordNet, a huge lexical database commonly used for natural language processing research. Watson seems fairly similar, if more sophisticated. I’m impressed both with my friend’s system and with Watson, but ultimately both are just trying to do fairly dumb semantic associations in a big database. It’s a lot of work and I don’t think that I could personally do it, but it’s a long way from any real understanding of natural language.

  • JoeLeTaxi

    Nemo,

    I found it so extraordinary that the only reasonable explanation I could think of was that, somehow, it was a forgery.

    If not, then I share your view that we have here an historical event of first magnitude.

  • combolek

    There is going to be a PBS piece about it on NOVA on Feb 9 (it will be repeated several times).

Leave a Reply