On September 25, I wrote an article in NRC Handelsblad, arguing that the Turing Test is outdated and that it's time to say farewell to the Turing Test as a way to decide whether or not machines can think.
[Use www.translate.google.com if you want to get an automatic translation of my NRC-article originally written in Dutch]
In the International Computer Games Association Journal, professor Jaap van den Herik, a specialist in artificial intelligence, refers to my article and seems to agree with me that it is time to say farewell to the Turing Test.
Here is the text of his contribution:
"FAREWELL TO THE TURING TEST?
Sixty years after Alan Turing’s seminal article in which he introduced the Imitation Game, the discussion on the scientific value of the game is still lively and challenging. Nowadays the Imitation Game is called the Turing Test. A modern and more practical form of this test is the Loebner Prize contest, which will have had its 20th edition on October 23, 2010.
It is safe to say that “passing the Turing Test” has served for sixty years as a paradigm for researchers in Artificial Intelligence, Cognitive Science, Philosophy, and Natural Language Processing. It is attractive to compare the outcome of the “thinking” process of a machine to the result of the thinking process of a human being. In the beginning (1950) such a machine was a one-processor engine and the contest was challenging for engineers and scientists. Gradually, the power of the machine increased and its storage capacities grew beyond the limits that are imposed by nature on human beings. (Here I admit that, in the future, technology may help human brains to augment the memory capabilities, but I disregard this development for the moment.)
The question I would like to address here is the following: is the Turing Test still relevant to our researchers? We assume that some of our readers will answer this question by ‘yes’. In particular, the Editorial Board may be expected to give this answer since we publish in this issue a contribution by Maarten van Emden and André Vellino, titled From Chinese Room to Human Window. Humanising the title would lead to: “From Searle to Michie and back again, with Turing as interested spectator.”
A fair Turing Test would be a test between equally capable entities. From 1950 to 1990 a test between a computer and a human being on their thinking capabilities in one or another area (or even in all areas of thinking) could be considered as a rather fair contest. However, since 1990 the information technology world has experienced accelerating progress in areas such as supercomputers, the Internet, multi-processor systems, and distributed multi-agent systems. Can we consider the Internet or a multi-agent system as an entity? If the answer is affirmative, then the next question is: can we consider such an entity as a suitable opponent for a human being in a Turing Test?
Recently, the Dutch journalist Bennie Mols discussed this topic in the Dutch daily paper NRC Handelsblad. In the article, the philosopher Johan van Benthem criticised the Turing Test by remarking that a network of computers is not the same as “a machine”. His argument was as follows. For a machine, we have a model (i.e., the Turing machine); for networks with many computers involved we lack an acceptable theoretical model. His second criticism was directed at the dichotomy of the outcome: it is either passing the Turing Test or not passing the Turing Test. There is no measure on the outcome, e.g., the machine scored reasonably well, or it achieved 40 per cent.
From the perspective of Artificial Intelligence and Cognitive Science, there are many more criticisms on the value of the Turing Test with respect to the current status of the technology. We mention (1) reactions on environmental input and (2) non-verbal communication.
In the same article, the Dutch scientist Frank van Harmelen states: “On the one hand, I believe that the Turing Test currently is an obstacle for scientific research to understand the very nature of artificial intelligence (...). On the other hand, I still consider a computer that passes the Turing Test as a fulfilment of the extreme dream of our research community, comparable to the extreme dream in physics on a Theory of Everything. We should keep that dream, but not take it as a guideline for our research.”
In the world of chatter bots, conversational systems, robots, and androids, much discussion is devoted to the value of the Turing Test. In the 2006 conference celebrating 50 years of Artificial Intelligence, participants could fill in a questionnaire on the “passing of the Turing test” (i.e., computers are able to mimic human intelligence). Forty-one per cent of the respondents believed that passing the test never happen. Another forty- one per cent believed it to be possible (but that it would take over fifty years to happen). Apparently, Ray Kurzweil is an exception (cf. Mols, 2010).
In the previous issue we discussed the notion of a paradigm shift with respect to minimax and MCTS. Simultaneously we were facing the fact that the Drosophila Melanogaster had lost its paradigm function. Your Editor believes that we are now at the point of saying Farewell to the Turing Test."
In the International Computer Games Association Journal, professor Jaap van den Herik, a specialist in artificial intelligence, refers to my article and seems to agree with me that it is time to say farewell to the Turing Test.
Here is the text of his contribution:
"FAREWELL TO THE TURING TEST?
Sixty years after Alan Turing’s seminal article in which he introduced the Imitation Game, the discussion on the scientific value of the game is still lively and challenging. Nowadays the Imitation Game is called the Turing Test. A modern and more practical form of this test is the Loebner Prize contest, which will have had its 20th edition on October 23, 2010.
It is safe to say that “passing the Turing Test” has served for sixty years as a paradigm for researchers in Artificial Intelligence, Cognitive Science, Philosophy, and Natural Language Processing. It is attractive to compare the outcome of the “thinking” process of a machine to the result of the thinking process of a human being. In the beginning (1950) such a machine was a one-processor engine and the contest was challenging for engineers and scientists. Gradually, the power of the machine increased and its storage capacities grew beyond the limits that are imposed by nature on human beings. (Here I admit that, in the future, technology may help human brains to augment the memory capabilities, but I disregard this development for the moment.)
The question I would like to address here is the following: is the Turing Test still relevant to our researchers? We assume that some of our readers will answer this question by ‘yes’. In particular, the Editorial Board may be expected to give this answer since we publish in this issue a contribution by Maarten van Emden and André Vellino, titled From Chinese Room to Human Window. Humanising the title would lead to: “From Searle to Michie and back again, with Turing as interested spectator.”
A fair Turing Test would be a test between equally capable entities. From 1950 to 1990 a test between a computer and a human being on their thinking capabilities in one or another area (or even in all areas of thinking) could be considered as a rather fair contest. However, since 1990 the information technology world has experienced accelerating progress in areas such as supercomputers, the Internet, multi-processor systems, and distributed multi-agent systems. Can we consider the Internet or a multi-agent system as an entity? If the answer is affirmative, then the next question is: can we consider such an entity as a suitable opponent for a human being in a Turing Test?
Recently, the Dutch journalist Bennie Mols discussed this topic in the Dutch daily paper NRC Handelsblad. In the article, the philosopher Johan van Benthem criticised the Turing Test by remarking that a network of computers is not the same as “a machine”. His argument was as follows. For a machine, we have a model (i.e., the Turing machine); for networks with many computers involved we lack an acceptable theoretical model. His second criticism was directed at the dichotomy of the outcome: it is either passing the Turing Test or not passing the Turing Test. There is no measure on the outcome, e.g., the machine scored reasonably well, or it achieved 40 per cent.
From the perspective of Artificial Intelligence and Cognitive Science, there are many more criticisms on the value of the Turing Test with respect to the current status of the technology. We mention (1) reactions on environmental input and (2) non-verbal communication.
In the same article, the Dutch scientist Frank van Harmelen states: “On the one hand, I believe that the Turing Test currently is an obstacle for scientific research to understand the very nature of artificial intelligence (...). On the other hand, I still consider a computer that passes the Turing Test as a fulfilment of the extreme dream of our research community, comparable to the extreme dream in physics on a Theory of Everything. We should keep that dream, but not take it as a guideline for our research.”
In the world of chatter bots, conversational systems, robots, and androids, much discussion is devoted to the value of the Turing Test. In the 2006 conference celebrating 50 years of Artificial Intelligence, participants could fill in a questionnaire on the “passing of the Turing test” (i.e., computers are able to mimic human intelligence). Forty-one per cent of the respondents believed that passing the test never happen. Another forty- one per cent believed it to be possible (but that it would take over fifty years to happen). Apparently, Ray Kurzweil is an exception (cf. Mols, 2010).
In the previous issue we discussed the notion of a paradigm shift with respect to minimax and MCTS. Simultaneously we were facing the fact that the Drosophila Melanogaster had lost its paradigm function. Your Editor believes that we are now at the point of saying Farewell to the Turing Test."
Jaap van den Herik
Reference
Mols, B. (2010). De computer kletst maar wat. NRC Handelsblad, Science, September 25 & 26, pp. 8-9.
Reference
Mols, B. (2010). De computer kletst maar wat. NRC Handelsblad, Science, September 25 & 26, pp. 8-9.
