How Intelligent are Intelligent Robots?

When giving talks about intelligent robots I've often been faced with the question "how intelligent is your robot?" with a tone of voice that suggests "...and should we be alarmed" It's a good question but one that is extremely difficult - if not impossible - to answer properly. I usually end up giving the rather feeble answer "not very", and I might well add "perhaps about as intelligent as a lobster" (or some other species that my audience will regard as reassuringly not very smart). I'm always left with an uneasy sense that I (and robotics in general) ought to be able to give an answer to this perfectly reasonable question. (Sooner or later I'm going to get caught out when someone follows up with "and exactly how intelligent is a lobster?")

Given that the study of Artificial Intelligence is over 60 years old, and that of embodied AI (i.e. intelligent robotics) not much younger, the fact that roboticists can't properly answer the question "how intelligent are intelligent robots" is, to say the least, embarrassing. It is I believe a problem that needs some serious attention.

Let's look at the question again. There is an implied abbreviation here: what my interlocutor means is: how intelligent are intelligent robots when compared with animals and humans? What's more we all assume a kind of 'scale' of intelligence - with humans (decidedly) at the top and, furthermore, a sense that a crocodile is smarter than a lobster, and a cat smarter than a crocodile. Where, then, would we place a robot vacuum cleaner, for instance, on this scale of animal intelligence?

Ok. To answer the question we clearly need to find a single measure, or test, for intelligence that is general enough it can be applied to robots, animals or humans. It needs to have a single scale broad enough to accommodate human intelligence and simple animals. This metric - let's call it GIQ for General (non-species-specific) Intelligence Quotient - would need to be extensible downwards - to accommodate single celled organisms (or plants for that matter) and of course robots because they're not very smart. Thinking ahead it should also be extensible upwards for super-human AI (which we keep being told is only a few decades away). Does such a measure exist already? No, I don't think it does, but I did come across this news posting on physorg.com a few days ago with the promising opening line How do you use a scientific method to measure the intelligence of a human being, an animal, a machine or an extra-terrestrial? It refers to a paper titled Measuring Universal Intelligence: Towards an Anytime Intelligence Test. I haven't been able to read the paper (it is behind a paywall) but - even from the abstract - it's pretty clear the problem isn't solved. In any event I'm doubtful because the news writeup talks of "interactive exercises in settings with a difficulty level estimated by calculating the so-called Kolmogorov complexity", which suggests a test that the agent being tested has to engage in. Well that's not going to work if you're testing the intelligence of a spider is it?

So let's set aside the problem of comparing the intelligence of robots with animals (or ET) for a moment. Are there existing non-species specific intelligence measures? This interesting essay by Jonathan Ball: The question of animal intelligence outlines several existing measures based on neural physiology. In summary they include:

• Encephalization Quotient (EQ): which measures whether the brain of a given species is bigger or smaller than would be expected, compared with that of other animals its size (winner: Humans)
• Cortical Folding: a measure based on the degree of cortical folding (winner: Dolphins)
• Connectivity: a measure based on comparing the average number of connections per neuron (winner: Humans)

Interestingly, if we take the connectivity measure - which Jonathan Ball suggests offers the greatest degree of correlation with intelligence - then if our robot is controlled by an artificial neural network we might actually have a common basis for comparison of human and robot intelligence.

So, even if none of them are entirely satisfactory it's clear that there has been a great deal of work on measures of animal intelligence. What about the field of robotics - are there intelligence metrics for comparing one robot with another (say a vacuum cleaning robot with a toy robot dog)? As far as I'm aware the answer is a resounding no. (Of course the same is not true in the field of AI where passing the Turing Test has become the iconic - if controversial - holy grail.)

But all of this presupposes, firstly, that we can agree on what we mean by 'intelligence' - which we do not. And secondly, that intelligence is a single thing that any one animal, or robot, can have more or less of* - which is also very doubtful.

---

*An observation made by an anonymous reviewer of one of my papers, for which I am very grateful.

Expecting the expected on Mars

Learned something new and surprising about Mars rovers (like Spirit and Opportunity, and the planned European rover ExoMars): that if little green Martians jumped up and down in front of the Rover's cameras we almost certainly wouldn't know it. There are two reasons: firstly, the communications links between the Mars rovers and Earth are intermittent and low-bandwidth, so you can't have a live video stream (webcam) from the Rover to Earth and, secondly, the Rover's onboard cameras have image processing software that is programmed to look for specific things, like interesting rocks. This means that the Rover simply wouldn't 'see' the Martians, they are - in a sense - programmed to expect the expected. Although we are used to seeing the amazing panoramic views from the surface of Mars, these still images are only grabbed infrequently so our Martian would have to be standing in front of the Rover at precisely the moment the image is captured for us to see him (it).

I just spent 2 days with a remarkably interesting group of space scientists (planetary geology, exobiology, etc), space industry and roboticists discussing the science and engineering of Mars sample return missions: i.e. to find, collect and then bring interesting Mars rocks back to Earth. Given the immense cost and technical risk of mounting such a mission it seems to me worth the extra small effort of giving the rover(s) systems that would allow them (and us) to notice unexpected or unusual stuff. An image processing module that, for instance, continuously looks for things in the camera's view that are the wrong colour, or shape, moving in a different way to everything else. The whole point of exploration is that you don't know what's there and, while I'm not suggesting there really are Martians (other than perhaps microbes), it does seem to me that we should engineer systems that allow for the possibility of discovering the unexpected.

What do Aliens look like?

Amazing meeting yesterday afternoon at the Science Museum. Here's the story: the British Association for the Advancement of Science (BA) has been asking for science questions via their web pages for awhile, in advance of Science Week, which is 7-16 March. A question that keeps coming up is "what do aliens look like?" so, to address that question, the BA pulled together a small panel which met yesterday. Our brief was to come up with some plausible alien life forms that can be visually presented during Science Week. The keyword here is plausible. It would be easy to pluck super-exotic aliens from the rich fauna of SF but then very difficult to explain the science. Of course, the fictional evolutionary science, or biochemistry, or ecosystems of even plausible aliens is going to have gaping holes, but we were tasked with trying to minimise those.

So, what did we come up with..? I can't say now but all will, I hope, be revealed during Science Week (and in this blog, then). ...and here is the press release describing our life forms.


Visualisation by Julian Hume, Research Fellow at the Natural History Museum/University of Portsmouth.