Artificially stupid ducks

“The Eugene Goostman chatbot passed the Turing test. So now we finally have real artificial intelligence.”

That is what was reported recently by many news outlets. Of course, it is horribly wrong.

A chatbot is not intelligent. A chatbot has no understanding of what it says. A chatbot simply delves into a database of previously stored sentences (usually automatically retrieved from the Internet), and loosely links them to what the person who is testing the bot is typing. It uses non sequiturs instead of actual answers, repeats a persons statements back at him, and switches from topic to topic without rhyme or reason.

The authors of Eugene Goostman gave their bot the backstory that it was a 13-year old boy from the Ukraine, whose native language was not English. The fact that he was supposed to be a foreigner was introduced to make members of the jury more forgiving of the irrational answers that the bot provided. The fact that he was supposed to be 13 years old was introduced to make members of the jury more forgiving of the nonsensical switching of topics and general lack of knowledge and understanding. If that cheap trick is considered acceptable, then we have had artificial intelligence for many years now.

I mean, I have a program wherein you can type any text that you want, and it will never respond. As such, it functions nicely as a replica of an autistic person. It would also be relatively easy to create a program that resembles a heavy sufferer of Tourettes.

But even if the authors had not coined up this backstory, and were still able to fool 10 out of 30 judges — would we then have to conclude that Eugene Goostman is ‘real’ artificial intelligence? Would Alan Turing conclude that?

The answer is “no”. The Turing Test is one of the most misrepresented tests in the history of science. It is not a litmus test for artificial intelligence. It is merely an illustration of a philosophical stance that Alan Turing took.

The issue is as follows: how can we know whether a computer is intelligent or not? When Turing was alive, this topic was hotly debated amongst computer scientists and philosophers. Some claimed that a computer can never be ‘really intelligent’, as you can examine its programs and databases and (theoretically) derive exactly how it produces its answers. The counter-argument is that you can also open up a person’s brain and (theoretically) derive exactly how that person produces his answers. So what features would you want a computer to have, which allow you to unequivocally state that it is ‘really’ intelligent?

Alan Turing’s answer was: it is not important what is inside the computer; what is important is its behavior. If a computer’s behavior is indistinguishable from an intelligence, we should conclude that it is intelligent. Even if we could open up the computer, look inside, and point out some features that make us say: “You see that? That is how that intelligent behavior is generated!” that would only teach us something about how intelligence comes about, and would not invalidate the computer as an intelligent being (unless we open up the computer and see a human inside who provides all the answers, of course).

The Turing Test is only an illustration of Turing’s philosophical principle. He says that if a computer can converse so well that you cannot distinguish it from a human, then the computer converses as well as a human, and thus converses intelligently. There is no stipulation like ‘conversing for only 5 minutes’ or ‘the computer is allowed to limit the topics’ or ‘the computer should be forgiven for bad English’. Such stipulations would make no sense, because an intelligent conversation should demonstrate an understanding of the world. A chatbot that does not at least encompass a model of the world can never demonstrate an understanding. Simply reflecting sentences that you pick off the Internet might fool some uninitiated people for a while (that is not too hard, ELIZA managed to fool Joseph Weizenbaum’s secretary in 1964), but it will fool nobody for longer stretches of time.

The whole point is that Turing wanted to introduce the Duck Test for artificial intelligence — if it looks, swims, and quacks like a duck, you should conclude that it is a duck. We now know that it is not hard to fool a couple of people for 5 minutes into thinking that just maybe that computer over there is actually a human. We can do that due to the enormous speed that computers have achieved in processing data, and the huge storage capacity that modern computers have. But despite the fact that, by itself, it is not an easy task to make people think that a computer is conversing like a 13-year old Ukrainian boy, succeeding at that task is not the same as succeeding at creating an artificial intelligence.

As written, the Turing Test is not a test of artificial intelligence. Turing’s principle, however, stands: the Duck Test is the only viable way of determining whether a computer is really intelligent. However, we should realize that the duck itself is much bigger and much more complex than Turing’s original illustration sketches.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: