Keeping tabs on Skynet

In line with the predictions of science fiction, computers are getting smarter. Now, scientists are on the way to devising a test to ascertain how close Artificial Intelligence (AI) is coming to matching wits with us, and if it’s drawing ahead.

Associate Professor David Dowe of Monash University’s Faculty of Information Technology, together with Dr Jose Hernandez-Orallo from Universitat Politecnica de Valencia in Spain have developed and conducted initial trials of a prototype Anytime Universal Intelligence test designed to gauge and compare the intelligence of humans, animals, machines, and, in principle, anything.

Both humans and an AI program known as Q-Learning undertook different versions of the test, with considerable work on adapting the interface necessary before animals can be tested. Despite not being a sophisticated program, Q-Learning scored competitively compared with the human participants.

Associate Professor Dowe said the ambiguity of the initial test results indicates the complexity of moving to a broader understanding of intelligence than the traditional method of using human intellect as the yardstick – a development necessary to determine if, or perhaps when, AI outstrips humans.

“We are using a mathematically-based definition of intelligence which is based, in simple terms, on the ability to detect patterns of various degrees of complexity.  In the future, the test should adapt to the user – becoming more complex if the user is scoring well, and more simple if the user is struggling,” said Associate Professor Dowe.

“Clearly, we have very specialised indications of the intelligence of computer programs, when they’re beating humans at activities like chess and the game show Jeopardy. We’re trying to establish a broader indication.

“With further research, this type of testing could help not only in assessing the progress of AI, but in driving development.”

Inspired partly by Foundation Chair in Computer Science at Monash University, Professor Chris Wallace’s research on Minimum Message Length, a theory of machine learning and statistics, Associate Professor Dowe has been working on alternatives to traditional measures of intelligence since the late 1990s. His projects have included the development of a relatively simple computer program that regularly scored close to the purported human average of 100 on standard IQ tests.

Associate Professor Dowe has been collaborating with Dr Hernandez-Orallo since 2003.

The Anytime Universal Intelligence project has attracted interest from the both the AI research community and the general public and has been featured most recently in New Scientist. Dr Hernandez-Orallo, recently presented the results of the testing at the Artificial General Intelligence Conference, hosted by Google, California.