< Failing the Turing Test


Friday, June 13, 2014

BROOKE GLADSTONE:  This past weekend, the tech media were all abuzz about a milestone moment in the history of computing and artificial intelligence. FOR THE FIRST TIME EV-ER, a computer program had passed the 65-year-old Turing Test.

CORRESPONDENT:  The computer program that’s called Eugene Goostman has passed the iconic Turing Test.

CORRESPONDENT:  If you’re worried about computers one day taking over Earth, you’re not gonna like this report, because a computer has done something no computers has ever done before, fooling judges into thinking it was a real person.

BROOKE GLADSTONE:  In order to pass the Turing Test, the computer must be mistaken for a human at least 30% of the time during a series of keyboard conversations with a live person. The University of Reading, which organized last Saturday's test, says that the Russian-made program Eugene Goostman, convinced 33% of the judges that it was human.

That the test took place on the 60th anniversary of Alan Turing's death only added to the poignancy of the achievement. What took away from the poignancy of the achievement was that, according to Techdirt's Mike Masnick, the whole thing was bogus, and everyone who reported on it should have known better.

MIKE MASNICK:  The press release came out of the University of Reading and it was from a professor named Kevin Warwick, who’s got a fairly long history of issuing these kinds of well-timed press releases that the press eats up. Right at the beginning of the press release, it actually says, “a supercomputer passed the Turing Test.” It's not a supercomputer at all. It's an application, it’s a chatbot. If you say something to it, it says something back to you. It’s not this brilliant breakthrough in artificial intelligence to make something that will have a conversation with you.

BROOKE GLADSTONE:  Another thing is it wasn't the first time anything passed the Turing Test right?

MIKE MASNICK:  It was just a few years ago there was a chatbot called Cleverbot that was entered into a competition, and it fooled almost 60% of the judges, which is an even higher scoring than this one had.

BROOKE GLADSTONE:  Something that puzzled me is that the designers of this particular chatbot told the judges that it was a 13-year-old Ukrainian boy.

MIKE MASNICK:  They did a social game. They, they played the judges by telling them that it was a 13-year-old boy from the Ukraine, so that the judges had a certain expectation for what the person they were talking to would say, so that that when it did say some very odd things in the actual chats, they just sort of brushed it off as, it must be, well, because he's not American or not British and, therefore, maybe it really was this Ukrainian boy.

BROOKE GLADSTONE:  Now, we can say that this whole episode was a kind of Turing Test for the media, right? Could they tell a real story from the fake one?


And, as you noted earlier, Professor Warwick is infamous for his claims to the press. It's apparently been going on for decades?

MIKE MASNICK:  In the late 90s and early 2000's, Professor Warwick became fairly famous for calling himself the very first cyborg because he had implanted a computer chip in his arm. It’s not clear that the computer chip ever did anything, but because he put it under his skin suddenly he was the world's first cyborg. And the press ate it up because what a great story.

A few years ago, there was another example, “the first human to contract a computer virus.”


What a great story. Everyone's gonna click on that and see what that’s about because everyone knows about computer viruses and the idea that they’re somehow morphing into humans is so exciting. For a while, someone kept a website up called Warwick Watch, which just detailed all of his crazy claims in the press and how the press was falling for it. The person who kept it going eventually decided to stop it because they said that Warwick himself was kind of using it as a list of his own personal accolades, [LAUGHS] –


- as he looked on it favorably, as opposed to what it was supposed to be doing, which was  mocking his crazy claims.

BROOKE GLADSTONE:  So a lot of media outlets ate this up, ABC and CNN. You have NBC. You have the Guardian, you have the Washington Post. But you also have tech sites that you’d think would know better - Gizmodo, CNET and Computerworld and Science Alert - all of which are echoing the claim that a supercomputer has passed the Turing Test or it’s an AI milestone, and so on.

MIKE MASNICK:  None of them questioning who set up the trial, how carefully was it set up, was it reviewed, who were the judges. All of the, the basic normal journalistic questions that you should ask when there’s a sort of fantastical claim about a milestone achievement, almost no one seemed to have done that.

BROOKE GLADSTONE:  Well, here’s the thing:  Reporters were being delivered a neatly-packaged bit of fun, on a weekend, from a respected institute of higher learning. What's the harm?

MIKE MASNICK:  When you're talking about reporting on an issue that could have major impact, in terms of computing and technology and how we interact with machines in our daily lives, you would hope that reporters would want to get that story right. There was also a lot of talk about, oh, are the computers taking over or should we be afraid? And the answer was always no because, of course, you shouldn’t be afraid. This was [LAUGHS] -


You know, this was a computer that acted like a 13-year-old boy, barely. It was an easy story and it happened on a weekend, something sort of fun to, to roll with that was gonna get a lot of attention, no matter what. It’s true of anything that is a major scientific breakthrough, where you have reporters who don't necessarily have the expertise in the area, who are very willing to take extraordinary claims and run with them.

BROOKE GLADSTONE:  But then why did Computerworld and Science Alert and Ars Technica and PC World and Yahoo Tech, and so on, report this wrong? Don’t they have the expertise?
MIKE MASNICK:  It's one of those things where what seems like a really sexy story is very tempting to just write it up and get it out, and the cost of getting it wrong doesn't seem that big, whereas, the benefit to getting the story up and getting all the page use from it, where everyone says, oh, you know, what a, what a great weekend story. I want to read about these crazy brilliant robots who are taking over the world, that’s a story that people like to get out there and make sure that they have their version of it.

BROOKE GLADSTONE:  So, what makes Techdirt so great? [LAUGHS]

MIKE MASNICK:  Not publishing on the weekend? [LAUGHS]

BROOKE GLADSTONE:  [LAUGHS] Mike, thank you very much.

MIKE MASNICK:  Thanks for having me.

BROOKE GLADSTONE:  Mike Masnick is the founder and the editor of Techdirt.


Mike Masnick

Hosted by:

Brooke Gladstone