Autonomous Robot Evolution: first challenges

Just spent an exciting two days at the first 'all hands meeting' of our new EPSRC funded project: Autonomous Robot Evolution (ARE): from cradle to grave (read here for an introduction). There are eleven of us in total: 4 postdocs (one from each partner university), 2 PhD students, 1 technician, and the four seniors (co-investigators).

Much of the meeting was spent discussing the fundamental (and tough) questions of (1) how we design the genotype, and the mapping between genotype and phenotype, and (2) how exactly we will physically create the robots. 

Let me outline where we are going with these two questions.

1. Genotype-phenotype mapping. As I explained here most evolutionary robotics research has, to date, used a direct mapping approach, in which each parameter of a robot's genome specifies one feature of the real robot (phenotype). For the robot's controller those parameters might be the weights of the robot's artificial neural network, and for the robot's body they might each specify some physical characteristic of the body (such as the length of leg segments in the illustration here). Of course in biology the mapping is indirect; to put it very simply, the genome determines how an organism develops, rather than the organism itself. And because the expression of genes is affected by the environment in which the organism is developing, identical genotypes give rise to non-identical phenotypes (albeit very similar as with identical twins); this is called phenotypic plasticity.

Because we are looking for both biological plausibility and phenotypic plasticity in this project, we have decided on an indirect mapping from genotype to phenotype. Exactly how this will work is still to be figured out, but I feel sure the genotype will need to be split into two parts: one for the robot's controller and the other for its body, and I rather suspect the mapping will be different for those two parts.

2. How to create the robots. In ARE we will adopt the engineering approach 'in which the process is embodied but takes place in a kind of evolution factory'. Now, in theory we could evolve every part of a robot's hardware, listed below.

But in practice this would be impossible; evolving any one of these subsystems would be a research project in its own right, and we're not attempting to re-run the whole of evolution in this project. Instead we will be designing and fabricating discrete modules for sensing, signalling, actuation and control, that we call 'organs'. So what will we actually evolve? It will be:

• the number, type and position of sensing, signalling and actuation subsystems, and
• the 3D shape of the robot’s physical structure or chassis.

At this point you're probably thinking: hang on a minute - if you're designing the organs then what's left to evolve? It's a fair question, but in fact evolution will still have huge freedom to choose which and how many organs and where to position them in the body. And when we bear in mind that we will be co-evolving the robot's controller then the space of all possible phenotypes is vast. Of course we may need to introduce some constraints: for instance that there must be at least one controller. But in general we want as few constraints as possible so that evolution is free to explore the phenotypic space to find the best robots, bearing in mind that we will be breeding robots to be able to operate in challenging environments.

And I would argue that in specifying and designing organs we have not compromised on biological plausibility at all. Biological evolution is, after all, highly modular. Most of the organs (and systems of organs) in your body were evolved long before hominids: livers, hearts, eyes, noses, vascular systems, digestive systems, central nervous systems; all of those evolved in early vertebrates (with some repurposing along the way*). Architecturally humans have a huge amount in common with all mammals. My dog is not so different from me (and in some aspects superior: her senses of hearing and smell are much better); our key differences are in morphology and intelligence. These are the two properties that we will be exploring through co-evolution in this project.

So, in the coming few months we have some big mechanical and electronic engineering challenges in this part of the project. Here are just a few:

• experiment with 3D printing materials and print heads,
• specify, design and prototype the organs (including their packaging and interconnects),
• decide on how to power the organs (i.e. a single central power organ, or a battery per organ) and figure out how to re-charge the batteries,
• determine how to connect the organs with the controller and each other (i.e. with wires or wirelessly), and
• work out the best way of picking and placing organs within the robot as it is 3D printed.

Challenging? For sure, but we have a wonderful team.

---

*See Neil Shubin's wonderful book Your Inner Fish.

Related blog posts:

Autonomous Robot Evolution: from cradle to grave

Robot Bodies and how to Evolve them

The Triangle of Life: Evolving Robots in Real-time and Real-space

Experiments in Artificial Theory of Mind

Since setting out my initial thoughts on robots with simulation-based internal models about 5 years ago - initially in the context of ethical robots - I've had a larger ambition for these models: that they might provide us with a way of building robots with artificial theory of mind - something I first suggested when I outlined the consequence engine 4 years ago.

Since then we've been busy experimentally applying our consequence engine in the lab, in a range of contexts including ethics, safety and imitation, giving me little time to think about theory of mind. But then, in January 2017 I was contacted by Antonio Chella, inviting me to submit a paper to a special issue on Consciousness in Humanoid Robots. After some hesitation on my part and encouragement on Antonio's I realised that this was a perfect opportunity.

Of course theory of mind is not consciousness but it is for sure deeply implicated. And, as I discovered while researching the paper, the role of theory of mind in consciousness (or, indeed of consciousness in theory of mind) is both unclear and controversial. So, this paper, written in the autumn of 2017, submitted January 2018, and - after tough review and major revisions - accepted in June 2018, is my first (somewhat tentative) contribution to the machine consciousness literature.

Experiments in Artificial Theory of Mind: From Safety to Story-Telling, advances the hypothesis that simulation-based internal models offer a powerful and realisable, theory-driven basis for artificial theory of mind.

Here is the abstract

Theory of mind is the term given by philosophers and psychologists for the ability to form a predictive model of self and others. In this paper we focus on synthetic models of theory of mind. We contend firstly that such models—especially when tested experimentally—can provide useful insights into cognition, and secondly that artificial theory of mind can provide intelligent robots with powerful new capabilities, in particular social intelligence for human-robot interaction. This paper advances the hypothesis that simulation-based internal models offer a powerful and realisable, theory-driven basis for artificial theory of mind. Proposed as a computational model of the simulation theory of mind, our simulation-based internal model equips a robot with an internal model of itself and its environment, including other dynamic actors, which can test (i.e., simulate) the robot’s next possible actions and hence anticipate the likely consequences of those actions both for itself and others. Although it falls far short of a full artificial theory of mind, our model does allow us to test several interesting scenarios: in some of these a robot equipped with the internal model interacts with other robots without an internal model, but acting as proxy humans; in others two robots each with a simulation-based internal model interact with each other. We outline a series of experiments which each demonstrate some aspect of artificial theory of mind.

For an outline of the work of the paper see the slides below, presented first at the SPANNER workshop in York in September 2018, then at a workshop on Social Learning and Cultural Evolution at ALife 2019 in July 2019.

In fact all of the experiments outlined here have been described in some detail in previous blog posts (although not in the context of artificial theory of mind):

1. The Corridor experiment 
2. The Pedestrian experiment
3. The Ethical robot experiments: with e-puck robots and with NAO robots
4. Experiments on rational imitation (the imitation of goals)
5. Story-telling robots**

The thing that ties all of these experiments together is that they all make use of a simulation-based internal model (which we call a consequence engine), which allows our robot to model and hence predict the likely consequences of each of its next possible actions, both for itself and for the other dynamic actors it is interacting with. In some of the experiments those actors are robots acting as proxy humans, so those experiments (in particular the corridor and ethical robot experiments) are really concerned with human-robot interaction.

Theory of mind is the ability to form a predictive model of ourselves and others; it's the thing that allows us to infer the beliefs and intentions of others. Curiously there are two main theories of mind: the 'theory theory' and the 'simulation theory'. The theory theory (TT) holds that one intelligent agent’s understanding of another’s mind is based on innate or learned rules, sometimes known as folk psychology. In TT these hidden rules constitute a 'theory' because they can be used to both explain and make predictions about others’ intentions.  The simulation theory (ST) instead holds that “we use our own mental apparatus to form predictions and explanations of someone by putting ourselves in the shoes of another person and simulating them” (Michlmayr, 2002).

When we hold our simulation-based internal model up against the simulation theory of mind, the two appear to mirror each other remarkably well. If a robot has a simulation of itself inside itself then it can explain and predict the actions of both itself, and others like itself by using its simulation-based internal model to model them. Thus we have an embodied computational model of theory of mind, in short artificial theory of mind.

So, what properties of theory of mind (ToM) are demonstrated in our five experiments?

Well, the first thing to note is that not all experiments implement full ST. In the corridor, pedestrian and ethical robot experiments robots predict their own actions using the simulation-based internal model, i.e. ST, but use a much simpler TT to model the other robots; we use a simple ballistic model for those other robots (i.e. by assuming the robot will continue to move at the speed and direction it is currently moving). Thus I describe these experiments as ST (self) + TT (other), or just ST+TT for short. I argue that this hybrid form of artificial ToM is perfectly valid, since you and I clearly don't model strangers we are trying to avoid in a crowded corridor as anything other than people moving in a particular direction and speed. We don't need to try and intuit their state of mind, only where they are going.

The rational imitation and story-telling experiments do however, use ST for both self and other, since a simple TT will not allow an imitating robot to infer the goals of the demonstrating robot, nor is it sufficient to allow a listener robot to 'imagine' the story told by the storytelling robot.

The table below summarises these differences and highlights the different aspects of theory of mind demonstrated in each of the five experiments.

*Theory Mode: ST (self) + TT/ST (other)

An unexpected real-world use for the approach set out in this paper, is to allow robots to explain themselves. I believe explainability will be especially important for social robots, i.e. robots designed to interact with people. Let me explain by quoting two paragraphs from the paper.

A major problem with human-robot interaction is the serious asymmetry of theory of mind. Consider an elderly person and her care robot. It is likely that a reasonably sophisticated near-future care robot will have a built-in (TT) model of an elderly human (or even of a particular human). This places the robot at an advantage because the elderly person has no theory of mind at all for the robot, whereas the robot has a (likely limited) theory of mind for her. Actually the situation may be worse than this, since our elderly person may have a completely incorrect theory of mind for the robot, perhaps based on preconceptions or misunderstandings of how the robot should behave and why. Thus, when the robot actually behaves in a way that doesn’t make sense to the elderly person, her trust in the robot will be damaged and its effectiveness diminished.

The storytelling model proposed here provides us with a powerful mechanism for the robot to be able to generate explanations for its actual or possible actions. Especially important is that the robot’s user should be able to ask (or press a button to ask) the robot to explain “why did you just do that?” Or, pre-emptively, to ask the robot questions such as “what would you do if I fell down?” Assuming that the care robot is equipped with an autobiographical memory, the first of these questions would require it to re-run and narrate the most recent action sequence to be able to explain why it acted as it did, i.e., “I turned left because I didn’t want to bump into you.” The second kind of pre-emptive query requires the robot to interpret the question in such a way it can first initialize its internal model to match the situation described, run that model, then narrate the actions it predicts it would take in that situation. In this case the robot acts first as a listener, then as the narrator (see slide 18 above). In this way the robot would actively assist its human user to build a theory-of-mind for the robot.

---

**This one remains, for the time-being, a thought experiment.

See also: When Robots Tell Each Other Stories: The Emergence of Artificial Fiction

Reference:

Michlmayr, M. (2002). Simulation Theory Versus Theory Theory: Theories Concerning the Ability to Read Minds. Master’s thesis, Leopold-Franzens- Universität Innsbruck.

The Pedestrian Experiment

Followers of this blog will know that I have been working for some years on simulation-based internal models - demonstrating their potential for ethical robots, safer robots and imitating robots. But pretty much all of our experiments so far have involved only one robot with a simulation-based internal model while the other robots it interacts with have no internal model at all.

But some time ago we wondered what would happen if two robots, each with a simulation-based internal model, interacted with each other. Imagine two such robots approaching each other in the same way that two pedestrians approach each other on the sidewalk. Is it possible that these 'pedestrian' robots might, from time to time, engage in the kind of 'dance' that human pedestrians do when one steps to their left and the other to their right only to compound the problem of avoiding a collision with a stranger? The answer, it turns out, is yes!

The idea was taken up by Mathias Schmerling at the Humboldt University of Berlin, adapting the code developed by Christian Blum for the Corridor experiment. Chen Yang, one of my masters students, has now updated Mathias' code and has produced some very nice new results.

Most of the time the pedestrian robots pass each other without fuss but in something between 1 in 5 and 1 in 10 trials we do indeed see an interesting dance. Here are a couple of examples of the majority of trials, when the robots pass each other normally, showing the robots' trajectories. In each trial blue starts from the left and green from the right. Note that there is an element of randomness in the initial directions of each robot (which almost certainly explains the relative occurrence of normal and dance behaviours).

And here is a gif animation showing what's going on in a normal trial. The faint straight lines from each robot show the target directions for each next possible action modelled in each robot's simulation-based internal model (consequence engine); the various dotted lines show the predicted paths (and possible collisions) and the solid blue and green lines show which next action is actually selected following the internal modelling.

Here is a beautiful example of a 'dance', again showing the robot trajectories. Note that the impasse resolves itself after awhile. We're still trying to figure out exactly what mechanism enables this resolution.

And here is the gif animation of the same trial:

Notice that the impasse is not resolved until the fifth turns of each robot.

Is this the first time that pedestrians passing each other - and in particular the occasional dance that ensues - has been computationally modelled?

All of the results above were obtained in simulation (yes there really are simulations within a simulation going on here), but within the past week Chen Yang has got this experiment working with real e-puck robots. Videos will follow shortly.

---

Acknowledgements.

I am indebted to the brilliant experimental work of first Christian Blum (supported by Wenguo Liu), then Mathias Schmerling who adapted Christian's code for this experiment, and now Chen Yang who has developed the code further and obtained these results.

Autonomous Robot Evolution: from cradle to grave

A few weeks ago we had the kick-off meeting, in York, of our new 4 year EPSRC funded project Autonomous Robot Evolution (ARE): cradle to grave. We - Andy Tyrrell and Jon Timmis (York), Emma Hart (Edinburgh Napier), Gusti Eiben (Free University of Amsterdam) and myself - are all super excited. We've been trying to win support for this project for five years or so, and only now succeeded. This is a project that we've been thinking, and writing about, for a long time - so to have the opportunity to try out our ideas for real is wonderful.

In ARE we aim to investigate the artificial evolution of robots for unknown or extreme environments. In a radical new approach we will co-evolve robot bodies and brains in real-time and real-space. Using techniques from 3D printing new robot designs will literally be printed, before being trained in a nursery, then fitness tested in a target environment (a mock nuclear plant). The genomes of the fittest robots will then be combined to create the next generation of ‘child' robots, so that – over successive generations – we will breed new robot designs in a process that mirrors the way farmers have artificially selected new varieties of plants and animals for thousands of years. Because evolving real robots is slow and resource hungry we will run a parallel process of simulated evolution in a virtual environment, in which the real world environment is used to calibrate the virtual world, and reduce the reality gap*. A hybrid real-virtual process under the control of an ecosystem manager will allow real and virtual robots to mate, and the child robots to be printed and tested in either the virtual or real environments.

The project will be divided into five work packages, each led by a different partner: WP1 Evolution (York), WP2 Physical Environment (UWE), WP3 Virtual Environment (York), WP4 Ecosystem Manager (Napier) and WP5 Integration and Demonstration (UWE).

Here in the Bristol Robotics Lab we will focus on work packages 2 and 5. The goal of WP 2 is the development of a purpose designed 3D printing system – which we call a birth clinic – capable of printing small mobile robots, according to a specification determined by a genome designed in WP1. The birth clinic will need to pick and place a number of pre designed and fabricated electronics, sensing and actuation modules (the robot’s ‘organs’) into the printing work area which will be over printed with hot plastic to form the complete robot. The goal of WP5 will be to integrate all components, including the real world birth clinic, nursery, and mock nuclear environment with the virtual environment (WP3) and the ecosystem manager (WP4) into a working demonstrator and undertake evaluation and analysis.

Here is an impression of what the birth clinic might look like

One of the most interesting aspects of the project is that we have no idea what the robots we breed will look like. The evolutionary process could come up with almost any body shape and structure (morphology). The same process will also determine which and how many organs (sensors, actuators, etc) are selected, and their positions and orientation within the body. Our evolved robot bodies could be very surprising indeed.

And who knows - maybe we can take a step towards Walterian Creatures?

---

*Anyone who uses simulation as a tool to develop robots is well aware that robots which appear to work perfectly well in a simulated virtual world often don't work very well at all when the same design is tested in the real robot. This problem is especially acute when we are artificially evolving those robots. The reason for these problems is that the model of the real world and the robot(s) in it inside our simulation is an approximation. The Reality Gap refers to the less-than-perfect fidelity of the simulation; a better (higher fidelity) simulator would reduce the reality gap.

Related materials

Article in de Volkskrant (in Dutch) De robotevolutie kan beginnen. Hoe? Moeder Natuur vervangen door virtuele kraamkamer (The robot evolution can begin. How? Replacing Mother Nature with virtual nursery), May 2018.

Eiben and Smith (2015) From evolutionary computing to the evolution of things, Nature.

Winfield and Timmis (2015) Evolvable Robot Hardware, in Evolvable Hardware, Springer.

Eiben et al. (2013) The Triangle of Life, European Conference on Artificial Life (ECAL 2013).

What is Artificial Intelligence? (Or, can machines think?)

Here are the slides from my York Festival of Ideas keynote yesterday, which introduced the festival focus day Artificial Intelligence: Promises and Perils.



I start the keynote with Alan Turing's famous question: Can a Machine Think? and explain that thinking is not just the conscious reflection of Rodin's Thinker but also the largely unconscious thinking required to make a pot of tea. I note that at the dawn of AI 60 years ago we believed the former kind of thinking would be really difficult to emulate artificially and the latter easy. In fact it has turned out to be the other way round: we've had computers that can expertly play chess for over 20 years, but we can't yet build a robot that could go into your kitchen and make you a cup of tea (see also the Wozniak coffee test).

In slides 5 and 6 I suggest that we all assume a cat is smarter than a crocodile, which is smarter than a cockroach, on a linear scale of intelligence from not very intelligent to human intelligence. I ask where would a robot vacuum cleaner be on this scale and propose that such a robot is about as smart as an e-coli (single celled organism). I then illustrate the difficulty of placing the Actroid robot on this scale because, although it may look convincingly human (from a distance), in reality the robot is not very much smarter than a washing machine (and I hint that this is an ethical problem).

In slide 7 I show how apparently intelligent behaviour doesn't require a brain, with the Solarbot. This robot is an example of a Braitenberg machine. It has two solar panels (which look a bit like wings) acting as both sensors and power sources; the left hand panel is connected to the right hand wheel and vice versa. These direct connections mean that Solarbot can move towards the light and even navigate its way through obstacles, thus showing that intelligent behaviour is an emergent property of the interactions between body and environment.

In slide 8 I ask the question: What is the most advanced AI in the world today? (A question I am often asked.) Is it for example David Hanson's robot Sophia (which some press reports have claimed as the world's most advanced)? I argue it is not, since it is a chatbot AI - with a limited conversational repertoire - with a physical body (imagine Alexa with a humanoid head). Is it the DeepMind AI AlphaGo which famously beat the world's best Go player in 2016? Although very impressive I again argue no since AlphaGo cannot do anything other than play Go. Instead I suggest that everyday Google might well be the world's most advanced AI (on this I agree with my friend Joanna Bryson). Google is in effect a librarian able to find a book from an immense library for you - on the basis of your ill formed query - more or less instantly! (And this librarian is poly lingual too.)

In slides 9 I make the point that intelligence is not one thing that animals, robots and AIs have more or less of (in other words the linear scale shown on slides 5 and 6 is wrong). Then in slides 10 - 13 I propose four distinct categories of intelligence: morphological, swarm, individual and social intelligence. I suggest in slides 14 - 16 that if we express these as four axes of a graph then we can (very approximately) compare the intelligence of different organisms, including humans. In slide 17 I show some robots and argue that this graph shows why robots are so unintelligent; it is because robots generally only have two of the four kinds of intelligence whereas animals typically have three or sometimes all four. A detailed account of these ideas can be found in my paper How intelligent is your intelligent robot?

In the next segment, slides 18-20 I ask: how do we make Artificial General Intelligence (AGI)? I suggest that the key difference between current narrow AI and AGI is the ability - which comes very naturally to humans - to generalise knowledge learned in one context to a completely different context. This I think is the basis of human creativity. Using Data from Star Trek the next generation as a SF example of an AGI with human-equivalent intelligence as what we might be aiming for in the quest for AGI I explain that there are 3 approaches to getting there: by design, using artificial evolution or by reverse engineering animals. I offer the opinion that the gap between where we are now and Data like AGI is about the same as the gap between current space craft engine technology and warp drive technology. In other words not any time soon.

In the fourth segment of the talk (slides 21-24) I give a very brief account of evolutionary robotics - a method for breeding robots in much the same way farmers have artificially selected new varieties of plants and animals for thousands of years. I illustrate this with the wonderful Golem project which, for the first time, evolved simple creatures then 3D printed the most successful ones. I then introduce our new four year EPSRC funded project Autonomous Robot Evolution: from cradle to grave. In a radical new approach we aim to co-evolve robot bodies and brains in real-time and real-space. Using techniques from 3D printing new robot designs will literally be printed, before being trained in a nursery, then fitness tested in a target environment. With this approach we hope to be able to evolve robots for extreme environments, however because the energy costs are so high I do not think evolution is a route to truly thinking machines.

In the final segment (slides 25-35) I return to the approach of trying to design rather than evolve thinking machines. I introduce the idea of embedding a simulation of a robot in that robot, so that it has the ability to internally model itself. The first example I give is the amazing anthropomimetic robot invented by my old friend Owen Holland, called ECCEROBOT. Eccerobot is able to learn how to control it's own very complicated and hard-to-control body by trying out possible movement sequences in its internal model (Owen calls this a 'functional imagination'). I then outline our own work to use the same principle - a simulation based internal model - to demonstrate simple ethical behaviours, first with e-puck robots, then with NAO robots. These experiments are described in detail here and here. I suggest that these robots - with their ability to model and predict the consequences of their own and others' actions, in other words anticipate the future - may represent the first small steps toward thinking machines.

---

Related blog posts:
60 years of asking can robot think?
How intelligent are intelligent robots?
Robot bodies and how to evolve them

Simulation-based internal models for safer robots

Readers of this blog will know that I've become very excited by the potential of robots with simulation-based internal models in recent years. So far we've demonstrated their potential in simple ethical robots and as the basis for rational imitation. Our most recent publication instead examines the potential of robots with simulation-based internal models for safety. Of course it's not hard to see why the ability to model and predict the consequences of both your own and others' actions can help you to navigate the world more safely than without that ability.

Our paper Simulation-Based Internal Models for Safer Robots demonstrates the value of anticipation in what we call the corridor experiment. Here a smart robot (equipped with a simulation based internal model which we call a consequence engine) must navigate to the end of a corridor while maintaining a safe space around it at all times despite five other robots moving randomly in the corridor - in much the same way you and I might have to navigate down a busy office corridor while others are coming in the opposite direction.

Here is the abstract from our paper:

In this paper, we explore the potential of mobile robots with simulation-based internal models for safety in highly dynamic environments. We propose a robot with a simulation of itself, other dynamic actors and its environment, inside itself. Operating in real time, this simulation-based internal model is able to look ahead and predict the consequences of both the robot’s own actions and those of the other dynamic actors in its vicinity. Hence, the robot continuously modifies its own actions in order to actively maintain its own safety while also achieving its goal. Inspired by the problem of how mobile robots could move quickly and safely through crowds of moving humans, we present experimental results which compare the performance of our internal simulation-based controller with a purely reactive approach as a proof-of-concept study for the practical use of simulation-based internal models.

So, does it work? Thanks to some brilliant experimental work by Christian Blum the answer is a resounding yes. The best way to understand what's going on is with this wonderful gif animation of one experimental run below. The smart robot (blue) starts at the left and has the goal of safely reaching the right hand end of the corridor – its actual path is also shown in blue. Meanwhile 5 (red) robots are moving randomly (including bouncing off walls) and their actual paths are also shown in red; these robots are equipped only with simple obstacle avoidance behaviours. The larger blue circle shows blue's 'attention radius' – to reduce computational effort blue will only model red robots within this radius. The yellow paths in front of the red robots in blue's attention radius show blue's predictions of how those robots will move (taking into account collisions with the corridor walls and with blue and each other). The light blue projection in front of blue shows which of the 34 next possible actions of blue that is internally modelled is actually chosen as the next action (which, as you will see, sometimes includes standing still).

What do the results show us? Christian ran lots of trials – 88 simulations and 54 real robot experiments – over four experiments: (1) the baseline in simulation – in which the blue robot has only a simple reactive collision avoidance behaviour, (2) the baseline with real robots, (3) using the consequence engine (CE) in the blue robot in simulation, and (4) using the consequence engine in the blue robot with real robots. In the results below (a) shows the time taken for the blue robot to reach the end of the corridor, (b) shows the distance that the blue robot covers while reaching the end of the corridor, (c) shows the “danger ratio” experienced by the blue robot, and (d) shows the number of consequence engine runs per timestep in the blue robot. The danger ratio is the percentage of the run time that anther robot is within the blue robot’s safety radius.

For a relatively small cost in additional run time and distance covered, panels (a) and (b), the danger ratio is very significantly reduced from a mean value of ~20% to a mean value of zero, panel (c). Of course there is a computational cost, and this is reflected in panel (d); the baseline experiment has no consequence engine and hence runs no simulations, whereas the smart robot runs an average of between 8 and 10 simulations per time-step. This is exactly what we would expect: predicting the future clearly incurs a computational overhead.

---

Full paper reference:

Blum C, Winfield AFT and Hafner VV (2018) Simulation-Based Internal Models for Safer Robots. Front. Robot. AI 4:74. doi: 10.3389/frobt.2017.00074

Acknowledgements:
I am indebted to Christian Blum who programmed the robots, set up the experiment and obtained the results outlined here. Christian lead authored the paper, which was also co-authored by my friend and research collaborator Verena Hafner, who was Christian's PhD advisor.

The imitation of goals

The imitation of goals is an extremely important form of social learning in humans. This is reflected in the early emergence of imitation in human infants; from the age of two, we humans can imitate both actions and their intended goals and this has been termed rational imitation.

Imitation has long been regarded as a compelling method for (social) learning in robots. However, robot imitation faces a number of challenges; one of the most fundamental is determining what to imitate. Although not trivial it is relatively straightforward to imitate actions — something we explored within the Artificial Culture project. But inferring goals from observed actions and thus determining which parts of a demonstrated sequence of actions are relevant, i.e., rational imitation, is very challenging.

The approach we take in our paper Rational imitation for robots: the cost difference model is to equip the imitating robot with a simulation-based internal model that allows the robot to explore alternative sequences of actions needed to attain the demonstrator robot’s potential goals (i.e., goals that are possible explanations for its observed actions). Comparing these actions with those observed in the demonstrator robot enables the imitating robot to infer the goals underlying those observed actions.

Here is the abstract from our paper:

Infants imitate behaviour flexibly. Depending on the circumstances, they copy both actions and their effects or only reproduce the demonstrator’s intended goals. In view of this selective imitation, infants have been called rational imitators. The ability to selectively and adaptively imitate behaviour would be a beneficial capacity for robots. Indeed, selecting what to imitate is one of the outstanding unsolved problems in the field of robotic imitation. In this paper, we first present a formalized model of rational imitation suited for robotic applications. Next, we test and demonstrate it using two humanoid robots.

My colleague Dieter Vanderelst conducted several experiments to demonstrate rational imitation. Let me outline one of them, which uses two NAO robots.

Here panels (a,d,g) show the setup with blue as the demonstrating robot and red the observing (then imitating) robot. Panels (b,e,h) Show trajectories of 3 runs of the demonstrator robot blue, and panels (c,f,i) show trajectories of 3 runs of the imitating robot red. Note that red always starts from the position it observes from, as you would if you were imitating your dance teacher.

In condition 1 blue moves directly to its goal position (panels a,b). Blue infers the goal is to move to red’s goal and does so directly in panel c.

In condition 2 blue deviates around an obstacle even though it has a direct path to its goal (panels d,e). In this case red infers that the deviation must be a sub-goal of blue — since blue is able to go directly to its goal but chooses not to — so in panel f red creates a trajectory via blue’s sub-goal. In other words red has correctly inferred blue’s intentions to imitate its goals.

In condition 3 blue’s path to its goal is blocked so it has no choice but to divert (panels g,h). In this case red infers that blue has no sub-goals and moves directly to the goal position (panel i).

---

Full paper reference:
Vanderelst, D. and Winfield, A. F. (2017) Rational imitation for robots: The cost difference model. Adaptive Behavior, 25 (2). pp. 60-71. Download pdf.

Acknowledgements:
The experiments here were conceived and conducted by Dr Dieter Vanderelst, within EPSRC project Verifiable Autonomy.

Why ethical robots might not be such a good idea after all

This week my colleague Dieter Vanderelst presented our paper: The Dark Side of Ethical Robots at AIES 2018 in New Orleans.

I blogged about Dieter's very elegant experiment here, but let me summarise. With two NAO robots he set up a demonstration of an ethical robot helping another robot acting as a proxy human, then showed that with a very simple alteration of the ethical robot's logic it is transformed into a distinctly unethical robot - behaving either competitively or aggressively toward the proxy human.

Here are our paper's key conclusions:

The ease of transformation from ethical to unethical robot is hardly surprising. It is a straightforward consequence of the fact that both ethical and unethical behaviours require the same cognitive machinery with – in our implementation – only a subtle difference in the way a single value is calculated. In fact, the difference between an ethical (i.e. seeking the most desirable outcomes for the human) robot and an aggressive (i.e. seeking the least desirable outcomes for the human) robot is a simple negation of this value.

On the face of it, given that we can (at least in principle) build explicitly ethical machines* then it would seem that we have a moral imperative to do so; it would appear to be unethical not to build ethical machines when we have that option. But the findings of our paper call this assumption into serious doubt. Let us examine the risks associated with ethical robots and if, and how, they might be mitigated. There are three.

1. First there is the risk that an unscrupulous manufacturer might insert some unethical behaviours into their robots in order to exploit naive or vulnerable users for financial gain, or perhaps to gain some market advantage (here the VW diesel emissions scandal of 2015 comes to mind). There are no technical steps that would mitigate this risk, but the reputational damage from being found out is undoubtedly a significant disincentive. Compliance with ethical standards such as BS 8611 guide to the ethical design and application of robots and robotic systems, or emerging new IEEE P700X ‘human’ standards would also support manufacturers in the ethical application of ethical robots. 
2. Perhaps more serious is the risk arising from robots that have user adjustable ethics settings. Here the danger arises from the possibility that either the user or a technical support engineer mistakenly, or deliberately, chooses settings that move the robot’s behaviours outside an ‘ethical envelope’. Much depends of course on how the robot’s ethics are coded, but one can imagine the robot’s ethical rules expressed in a user-accessible format, for example, an XML like script. No doubt the best way to guard against this risk is for robots to have no user adjustable ethics settings, so that the robot’s ethics are hard-coded and not accessible to either users or support engineers. 
3. But even hard-coded ethics would not guard against undoubtedly the most serious risk of all, which arises when those ethical rules are vulnerable to malicious hacking. Given that cases of white-hat hacking of cars have already been reported, it's not difficult to envisage a nightmare scenario in which the ethics settings for an entire fleet of driverless cars are hacked, transforming those vehicles into lethal weapons. Of course, driverless cars (or robots in general) without explicit ethics are also vulnerable to hacking, but weaponising such robots is far more challenging for the attacker. Explicitly ethical robots focus the robot’s behaviours to a small number of rules which make them, we think, uniquely vulnerable to cyber-attack.

Ok, taking the most serious of these risks: hacking, we can envisage several technical approaches to mitigating the risk of malicious hacking of a robot’s ethical rules. One would be to place those ethical rules behind strong encryption. Another would require a robot to authenticate its ethical rules by first connecting to a secure server. An authentication failure would disable those ethics, so that the robot defaults to operating without explicit ethical behaviours. Although feasible, these approaches would be unlikely to deter the most determined hackers, especially those who are prepared to resort to stealing encryption or authentication keys.

It is very clear that guaranteeing the security of ethical robots is beyond the scope of engineering and will need regulatory and legislative efforts. Considering the ethical, legal and societal implications of robots, it becomes obvious that robots themselves are not where responsibility lies. Robots are simply smart machines of various kinds and the responsibility to ensure they behave well must always lie with human beings. In other words, we require ethical governance, and this is equally true for robots with or without explicit ethical behaviours.

Two years ago I thought the benefits of ethical robots outweighed the risks. Now I'm not so sure. I now believe that - even with strong ethical governance - the risks that a robot’s ethics might be compromised by unscrupulous actors are so great as to raise very serious doubts over the wisdom of embedding ethical decision making in real-world safety critical robots, such as driverless cars. Ethical robots might not be such a good idea after all.

*As a footnote let me explain what I mean by explicitly ethical robots: these are robots that select behaviours on the basis of ethical rules - in a sense they can be said to reason about ethics (in our case by evaluating the ethical consequences of several possible actions). Here I'm using the terminology of James Moor, who proposed four kinds of ethical agents, as I explain here. Moor shows in his classification that all robots (and AIs) are ethical agents in the sense that they can all have an ethical impact.

Thus, even though we're calling into question the wisdom of explicitly ethical robots, that doesn't change the fact that we absolutely must design all robots to minimise the likelihood of ethical harms, in other words we should be designing implicitly ethical robots within Moor's schema.

Here is the full reference to our paper:

Vanderelst D and Winfield AFT (2018), The Dark Side of Ethical Robots, AAAI/ACM Conf. on AI Ethics and Society (AIES 2018), New Orleans.

---

Related blog posts:
The Dark side of Ethical Robots
Could we make a moral machine?
How ethical is your ethical robot?
Towards ethical robots: an update
Towards an Ethical Robot

Ethical Governance: what is it and who's doing it?

These days I often find myself talking about ethical governance. Not just talking about but advocating: for instance in written evidence to the 2016 parliamentary select committee on robots and AI I made the link between ethical governance and trust. I believe that without transparent ethical governance robotics and AI will not win public trust, and without trust we will not see the societal benefits of robots and AI that we all hope for.

But what exactly is ethical governance and who is doing it, and perhaps more importantly, who in robotics and AI is doing it well?

In a draft paper on the subject I define ethical governance as

a set of processes, procedures, cultures and values designed to ensure the highest standards of behaviour. Ethical governance thus goes beyond simply good (i.e. effective) governance, in that it inculcates ethical behaviours. Normative ethical governance is seen as an important pillar of responsible research and innovation (RRI), which “entails an approach, rather than a mechanism, so it seeks to deal with ethical issues as or before they arise in a principled manner rather than waiting until a problem surfaces and dealing with it in an ad hoc way [1]” 

The link I make here between ethical governance and responsible research and innovation is I think really important. Ethical governance is a key part of RRI. They are not the same thing but it would be hard to imagine good ethical governance without RRI, and vice versa.

So what would I expect of companies or organisations who claim to be ethical? As a starting point for discussion here are five things that ethical companies should do:

• Have an ethical code of conduct, so that everyone in the company understands what is expected of them. This should sit alongside a mechanism which allows employees to be able to raise ethical concerns, if necessary in confidence, without fear of displeasing a manager.
• Provide ethics training for everyone, without exception. Ethics, like quality, is not something you can do as as add-on; simply appointing an ethics manager, while not a bad idea, is not enough. Ethical governance needs to become part of a company's culture and DNA, not just in product development but in management, finance, HR and marketing too.
• Undertake ethical risk assessments of all new products, and act upon the findings of those assessments. A toolkit, or method, for ethical risk assessment of robots and robotic systems exists in British Standard BS 8611, which - alongside much else - sets out 20 ethical risks and hazards together with recommendations on how to mitigate these and verify that they have been addressed.
• Be transparent about your ethical governance. Of course your robots and AIs must be transparent too, but here I mean transparency of process, not product. It's not enough to claim to be ethical, you need to show how you are ethical. That means publishing your ethical code of conduct, membership of your ethics board if you have one (and its terms of reference), and ideally case studies showing how you have conducted ethical risk assessments.
• Really value ethical governance.  Even if you have the four processes above in place you also needs to be sincere about ethical governance; that ethical governance is one of your core values, and just not a smokescreen for what you really value, like maximising shareholder returns.

My final point about really valuing ethical governance is of course hard to evidence. But, like trust, confidence in a company's claim to be ethical has to be earned and - as we've seen - can easily be damaged.

This brings me to my second question: who is doing ethical governance? And are there any examples of best practice? A week or so ago I asked Twitter this question. I've had quite a few nominations but haven't yet looked into them all. When I have, I will complete this blog post.

---

[1] Rainey, S., and Goujon, P. (2011). Toward a Normative Ethical of Governance of Technology. Contextual Pragmatism and Ethical Governance. In Ren von Schomberg (ed.) Towards Responsible Research and Innovation in the Information and Communication Technologies and Security Technologies Fields, Report of the European Commission-DG Research and Innovation.