How to make an ethical robot, and why we probably shouldn't

On Thursday 5th June I gave a keynote talk for workshop Machines with Morals: interdisciplinary perspectives organised by Rebecca Raper and Oliver Bridge at Cranfield University.

Slide 2
 

Robot ethics and machine ethics are two sides of the same coin. Robot ethics are ethics for humans. Machine ethics are ethics for robots.

Slide 3

But robot ethics and machine ethics are at very different levels of urgency and maturity. Robot Ethics is a much more pressing concern, given the rapid pace of developing applications as diverse as driverless cars, assisted living robots and smart robot toys. Also robot ethics has a large and active community which is already making progress toward standards and policy.

In contrast machine ethics remains the subject of basic research by a very small community of scholars. There are in fact no real-world ethical robots at the time of writing and it seems unlikely that there will be for some years.

Slide 4
 

Wendell Wallach and Colin Allen, in their wonderful 2009 book posed the open question: “Do we have a moral imperative to try and build ethical robots?” and suggest that the answer is (a qualified) yes.

Slide 5

James Moor’s important and influential paper 2006 set out a set of four categories of moral agency, from none to full.

Examples of ethical impact agents are kitchen knives and hammers. Both can be evaluated for ethical use (i.e. surgery) and unethical use (i.e. stabbing someone).

An example implicit ethical agent is the kind of blunt plastic knife that comes with airline food.

An explicit ethical agent can reason about ethics. Very few explicit agents have been demonstrated, not least because they are very hard to build.

The only full ethical agents we know of are adult humans of sound mind.

Slide 6

Allen, Smit and Wallach defined 3 approaches to explicit ethical machines in their 2005 paper. A training approach, which they call top down; a constraint (rules) based approach which they call bottom up and a hybrid approach that combines the two.

The work I will describe in this talk is all bottom-up. I know of only one instance of a top-down approach. The wonderful work of Susan and Michael Anderson: see their paper shown here.

Slide 7

Is it possible to build a moral machine: a robot capable of choosing or moderating its actions on the basis of ethical rules? Until 2014 I thought the idea impossible. But I changed my mind. In fact, also developed and experimentally tested an ethical robot. What brought about this U-turn?

Slide 8

 

First was thinking about very simple ethical behaviours. Imagine you see someone not looking where they’re going - about to walk into a hole in the pavement. Most liklely you will intervene. But why? It’s not just because you’re a good person – you also have the cognitive machinery to predict the consequences of someone’s actions.

Slide 9

Now imagine it’s not you, but a robot with four possible next actions. From the robot's perspective, it has two safe options: standstill (A), or turn to its left (B). But if the robot can model the consequences of both its own actions and the human's - another possibility opens up: the robot could choose to collide with the human to prevent him from falling into the hole (action D).

Slide 10

Let’s write this down as a logical rule. Remarkably the rule appears to match Asimov’s first law of robotics: A robot may not injure a human being or, through inaction, allow a human being to come to harm. The through inaction clause is important because it allow the robot to be morally proactive.

Slide 11

So emerged the idea is that we might be able to build a robot with Asimovian ethics. We need to equip the robot with the ability to predict the consequences of both its own, and other(s) actions, plus the hard-wired ethical rule in the previous slide.

Slide 12

Then came the realisation that the technology we need not only exists but is mature and commonplace in robotics research – it is the robot simulator. Robot simulators provide developers with a virtual environment for prototyping robot code before then running that code on the real robot.

Slide 13

But a robot simulator is not enough on its own. It also needs to be running inside the ethical robot. Thus, we set about designing a simulation-based internal model, which we call a consequence engine, shown here. On the right had the vertical green line describes the Sense Plan Act control system of most robots.

The consequence engine runs in parallel. The internal simulator has the three components shown here: a world model (with physics), a robot model (of itself and others), and an exact copy of the robot’s real controller.

For the current disposition of the robot – and others – the CE loops through all next possible actions, in order to estimate what might happen for each action. Then all of those predictions are evaluated, and the safety or ethics logic modifies the real robot’s action selection. Our robots are typically able to loop through 30 next possible actions every half a second.

Slide 14

The action evaluator codes the estimated outcome for each robot’s action (and the proxy human robot), on a scale of 0 to 10. Where 0 is completely safe and 10 is very dangerous. The value 4 codes for a collision; in reality simple obstacle avoidance, so no collision at all.

This simple table shows this mechanism assuming just 4 next possible actions. These numerical values allow the ethical robot to choose ‘ahead right’ as the least unsafe outcome for the proxy human. The lowest combined outcome values.

Slide 15
 

We built an ethical robot based on these ideas. We don’t have a real hole in the ground – just a danger zone, and we use robots as proxy humans. We ran two sets of experiments first with e-puck then with NAO robots. Let me show you these results – testing a simple Asimovian robot.

Slide 16

This short movie clip shows the robots of trial 2*. The ethical A-robot starts at the lower middle and the proxy-human H-robot starts from the left. The first run is in real time, then successive runs are speeded up.

Notice especially the moment when the A-robot ‘notices’ the H-robot is heading for danger and diverts from its path to intercept it. This is when Asimov’s 1st law is triggered.

We see the A-robot successfully prevents the H-robot from falling into the hole in every run. 

 

To the best of my knowledge this was the world's first demonstration of an explicitly ethical robot.

*Trial 1 is simply the ethical robot avoiding the hole.

Slide 17

After running trial 2 with the e-puck robots we decided to test our Asimovian robot with an ethical dilemma by introducing a second proxy-human H2 – also heading toward danger. As far as we know this is the world’s first experimental demonstration of an ethical robot facing a balanced dilemma.

Trial 3 is very interesting because on many runs the A-robot is seen to ‘dither’. We see this on the first run when the A-robot could have easily reached H2 to intercept it, but failed to do so, resulting in both H and H2 falling into the hole.

Because the consequence engine is running continuously, the A-robot can change its decision every half a second. This explains the dithering we observe here.

Paris conference on Safe and Ethical AI

Earlier this month I was privileged to part of the inaugural conference of the International Association for Safe and Ethical AI. The two day conference was held in Paris, on February 6^th and 7^th, and hosted by the OECD.  The timing and location of the conference was arranged to directly precede the governmental AI summit on February 10^th and 11^th. I was one of around 650 invited representatives from academia, civil society, industry, media, and government. It was a remarkable meeting, with terrific keynote talks, including three from Nobel prize winners, Geoffrey Hinton, Maria Ressa and Joseph Stiglitz.  

As someone who has been worrying about robot and AI ethics for longer than most who attended I was *very* pleased that there was a strong consensus around the need for regulation, supported by standards, alongside urgent concerns over the huge energy and water costs of AI that are completely at odds with sustainable development goals.

The conference concluded by publishing a Call to Action for lawmakers, academics, and the public ahead of the AI Summit, with ten critical action items. Overall, the action items are very good. I’m especially pleased to see ‘mandatory reporting of incidents’ in action 5. This is something I lobbied for. My one disappointment however is that the call for action statement has no explicit mention of the need to mitigate the energy costs of AI.

Here below are a few photos from the conference.

A slide from Joseph Stiglitz’ wonderful keynote: AI and Economic Risk: Assessment and Mitigation.

 

A slide from Kate Crawford’s excellent keynote: Hyperscaled: The Global Challenge of Sustainability in AI.

  

Me with Oxford colleague Pericle Salvini presenting our RoboTIPS work on accident investigation.

New paper: A Simulated real-world upper-body Exoskeleton Accident and Investigation

Back in February I posted a very brief account of our third RoboTIPS simulated accident and investigation, centred on an upper-body exoskeletion in an industrial setting. Since then we've published a paper with a full account. My colleague Pericle Salvini presented the paper at the 9th International Conference on Robot Ethics and Standards (ICRES 2024), last week.

Here is the paper abstract:

This paper describes the enactment of a simulated (mock) accident involving an upper-body exoskeleton and its investigation. The accident scenario is enacted by role-playing volunteers, one of whom is wearing the exoskeleton. Following the mock accident, investigators – also volunteers – interview both the subject of the accident and relevant witnesses. The investigators then consider the witness testimony alongside robot data logged by the ethical black box, in order to address the three key questions: what happened?, why did it happen?, and how can we make changes to prevent the accident happening again? This simulated accident scenario is one of a series we have run as part of the RoboTIPS project, with the overall aim of developing and testing both processes and technologies to support social robot accident investigation.

 The paper sets out, for the first time, the experimental method we have developed:

1. The accident scenario is enacted by human volunteers, role playing the subject of the accident, together with both direct  and indirect witnesses. The subject is the person to whom the accident happens. Direct witnesses are those who either witness or discover the accident, and indirect witnesses are those who might be supervisors or managers of the subject and/or the facility, or representatives of the robot's manufacturer. 
2. Prior to the enactment the project team brief the volunteers. Each briefing is specific to the role and, with the exception of the subject, volunteers are briefed only on their role, and not the whole scenario. This is so that they witness the accident (or it's aftermath) for the first time during the enactment. Only the subject is fully briefed on the scenario, including the safety aspects explained below, so that they are confident that they will not come to harm or be fearful during the enactment.
3. The enactment is stage managed by project team members. Although the simulation resembles a piece of theatre, volunteers are not asked to learn any lines. Apart from any specific action essential to the scenario (which will be prompted by the stage manager) the volunteers are invited to ad lib in a way that is appropriate to the roles they are playing. Volunteers are asked to wait in a side room until they are called a few moments before they are needed.
4. Safety of the volunteers, and especially the subject, is of paramount importance. Thus, if the scenario simulates physical harm to the subject, then – when the accident happens – the enactment is briefly suspended by the stage manager and the subject is helped into the position they might be expected to be in, following the accident. The project team conduct a safety risk assessment and if necessary modify the scenario and/or its stage management to mitigate any risks and the simulation is only undertaken after university research ethics approval.
5. The accident investigators are also volunteers and, ideally, the lead accident investigator has expertise and/or experience in accident investigation. Robotics expertise is not essential, as the aims and process of investigation are common to all accident or incident (near miss) investigations. The accident investigators are not briefed on the scenario, only the type of robot involved. Necessarily the accident investigators are not present during the enactment of the simulated accident. To reduce the time burden on all volunteers we stage the accident and its investigation on a single day, with the accident investigators arriving after the enactment. 

We were very lucky indeed that University of Nottingham Prof Carl McRae genrously acted as lead investigator for all three accident simulations in RoboTIPS. Carl is an authority on accident investigation in both aviation and heathcare. This meant that the process that Carl, together with a second volunteer investigator, followed asked the same questions that a real investigation would ask, namely: what happened, why did it happen, and how can we improve the system so that it doesn't happen again.

The full paper is on ArXiv here: https://arxiv.org/pdf/2411.14008v1

A simulated upper body exoskeleton accident and investigation

On Wednesday 21 February we ran the third of our RoboTIPS simulated accident scenarios in the Bristol Robotics Lab. This scenario focussed on an upper-body exoskeleton in an industrial environment.



Above left we see Dan working to move boxes, with the physical support of the wonderful Tribonix exoskeleton. On the right Dan has fallen to the floor, attended by his manager Monica and paramedic Ben. The simulation was carefully scripted and stage managed to ensure that none of the volunteers were hurt or, indeed, ever at risk.

 

Following the simulation the accident was investigated by lead investigator Carl and co-investigator Jack. Carl Macrae is a leading authority on accident investigation. Here we see Jack and Carl interviewing expert witness Appolinaire, observed by RoboTIPS project lead Marina Jirotka.

In addition to witness testimony our investigators were also able to examine Ethical Black Box data logs collected from the exoskeleton during the simulated accident.

The simulated accident scenario was a huge success. The various roles (not all of which are shown in the photos here) were acted brilliantly by our volunteers Dan Read, Ashwin Chandapur, Monica Monica, Surin Machaiah, Ben Allen and Dr Appolinaire Etoundi. And despite a complicated scenario which included human-human as well as human-robot interaction, our accident investigators Prof Carl Macrae and Jack Hughes were able to deduce, with reasonable accuracy, what happened and why. We are especially grateful to Romain Derval and Filip Hanus, co-founders of Tribonix, for both kindly agreeing to the use of their exoskeleton and generously working with RoboTIPS during the planning and enactment of this simulation.

The simulation was subject to Research Ethics Committee approval CATE-2324-218.

---

See also: 

Our first mock social robot accident and investigation

Robot Accident Investigation 

Our first mock social robot accident and investigation

Robot accidents are inevitable. These days the likelihood of serious accidents involving industrial robots is pretty low (but not zero), because such robots are generally inside safety cages. But a newer generation of social robots - robots designed to interact directly with people, including vulnerable elderly people or children - means that accidents are now much more likely. And if we also take into account ethical harms alongside physical harms, then the potential for accidents increases still further. Psychological harms include addiction, over trusting, or deception, and societal harms include privacy violations. For more on these ethical harms see my blog post outlining an ethical risk assessment of a smart robot teddy bear.

It has puzzled me for some years that there has been almost no research on robot accident investigation. In the RoboTIPS project we are addressing this deficit by developing both the technology - which we call an Ethical Black Box (EBB) - and the processes of robot accident investigation. One of the most exciting aspects of RoboTIPS is that we're running a series of mock, i.e. staged, social robot accidents in order to road test the EBB and investigation processes in as close to a real situation as is feasible in a research project. RoboTIPS started in March 2019, but then just as we were ready to trial our first mock accident the Covid pandemic hit, and closed down the lab.

So it was great that last week we finally managed to run the a pilot of our first (of three) mock accident scenarios. The scenario, based around an assisted living robot helping an elderly person to live independently, was sketched out in late 2019, and then - during the lockdown - rehearsed in a number of online events, including a podcast radio play for Oxford Sparks and CSI Robot during the UKRAS Festival of Robotics 2021.

Here is the scenario:

Imagine that your elderly mother, or grandmother, has an assisted living robot to help her live independently at home. The robot is capable of fetching her drinks, reminding her to take her medicine and keeping in touch with family. Then one afternoon you get a call from a neighbour who has called round and sees your grandmother collapsed on the floor. When the paramedics arrive they find the robot wandering around apparently aimlessly. One of its functions is to call for help if your grandmother stops moving, but it seems that the robot failed to do this

To enact this scenario we needed a number of volunteers: one to act as Rose - the subject of the accident, a second as the neighbour who discovers the accident and raises the alarm, a third as the paramedic who attends to Rose, a fourth who acts in the role of the cleaner and a fifth in the role of manager of the group of homes in which Rose lives. We also needed volunteers to act as members of the accident investigation team who are called in to try and discover what happened, why it happened and, if possible, what changes need to be made to how to ensure the accident doesn't happen again.

This is the mock accident taking place in the kitchen of our assisted living studio. Left shows the neighbour, acted by Paul, discovering Ross, acted by Alex, injured on the floor. (Note the chair on its side.) Right is the paramedic, role-played by Luc, attending to Ross. Meanwhile the Pepper robot is moving around somewhat aimlessly.

Our brilliant Research Fellow Dr Anouk van Maris, who organised the whole setup, persuaded five colleagues from the Bristol Robotics Lab. All were male, so Rose became Ross. Only one volunteer: Alex, who played the part of Ross, was fully briefed. The other four role played brilliantly and, although they were briefed on their roles, they were not told what was going to happened to Ross, or the part the Pepper robot played (or maybe didn't play) in the accident. Two colleagues from Oxford, Lars and Keri, kindly volunteered to act as the accident investigators. Lars and Keri also had no prior knowledge of the circumstances of the accident, and had to rely on (i) inspecting the robot and the scene of the accident, (ii) the data from the robot's EBB, and (iii) testimonies from Ross, the neighbour, the paramedic, the cleaner and the facility manager.

Here we see Lars interviewing Medhi, who acted as the house manager, while Ben, acting as the cleaner, waits to be interviewed. Inside the studio Keri is interviewing the neighbour and parademic.

So, what were the findings of our accident investigators? They did very well indeed. Close examination of the EBB data, alongside consideration of the (not always reliable) witness testimony enabled Lars and Keri to correctly deduce the role that the robot played in the accident. They were also able to make several recommendations on operational changes.  But I will not reveal their findings in detail here as we intend to run the same mock accident again soon with a different set of volunteers and - in case any of them should read this blog - I don't want to give the game away!

Acknowledgements

Very special thanks to Dr Anouk van Maris. Also Dr Pericle Salvini, who worked with Anouk in finalising the detail of the scenario and during the pilot itself. Also, huge thanks to BRL volunteers Dr Alex Smith, Dr Paul Bremner, Dr Luc Wijnen, Mehdi Sobhani and Dr Ben Ward-Cherrier. And last but not least a very big thank you to Dr Lars Kunze, Oxford Robotics Institute and Keri Grieman, Dept of Computer Science, Oxford.

From the left: Pericle, Ben, Lars, Alex, Keri, Medhi, Paul, Anouk, Luc, Lola and me. Pepper is looking nervously at Lola.

On Sustainable Robotics

The climate emergency brooks no compromise: every human activity or artefact is either part of the solution or it is part of the problem. 

I've worried about the sustainability of consumer electronics for some time, and, more recently, the shocking energy costs of big AI. But the climate emergency has also caused me to think hard about the sustainability of robots. In recent papers we have defined responsible robotics as

... the application of Responsible Innovation in the design, manufacture, operation, repair and end-of-life recycling of robots, that seeks the most benefit to society and the least harm to the environment.

I will wager that few robotics manufacturers - even the most responsible - pay much attention to repairability and environmental impact. And, I'm ashamed to say, very little robotics research is focused on the development of sustainable robots. A search on google scholar throws up just a handful of great papers detailing work on upcycled and sustainable robots (2018), sustainable robotics for smart cities (2018), and sustainable soft robots (2020).

I was then delighted when, a few weeks ago, my friend and colleague Michael Fisher, drafted a proposal for a new standard on Sustainable Robotics. The proposal received strong support from the BSI robotics committee. Here is the formal notice requesting comments on Michael's proposal: BS XXXX Guide to the Sustainable Design and Application of Robotic Systems. Anyone can comment (although you do need to register first). The deadline is 1 April 2021. 

So what would make a robot sustainable? In my view it would have to be:

1. Made from sustainable materials. This means the robot should, as far as possible, use recycled materials (plastics or metals), or biodegradable materials like wood. Any new materials should be ethically sourced. 
2. Low energy. The robot should be designed to use as little energy as possible. It should have energy saving modes. If an outdoor robot then it should use solar cells and/or hydrogen cells when they become small enough for mobile robots. Battery powered robots should always be rechargeable. 
3. Repairable. The robot would be designed for ease of repair, using modular, replaceable parts as much as possible - especially the battery. Additionally the manufacturers should provide a repair manual so that local workshops could fix most faults. 
4. Recyclable. Robots will eventually come to the end of their useful life, and if they cannot be repaired or recycled we risk them being dumped in landfill. To reduce this risk the robot should be designed to make it easy to re-use parts, such as electronics and motors, and re-cycle batteries, metals and plastics.
   

These are, for me, the four fundamental requirements, but there are others. The BSI proposal adds the environmental effects of deployment (it is unlikely we would consider a sustainable robot designed to spray pesticides as truly sustainable), or of failure in the field. Also the environmental effect of maintenance; cleaning materials, for instance. The proposal also looks toward sustainable, upcyclable robots as part of a circular economy.

This is Ecobot III, developed some years ago by colleagues in the Bristol Robotics Lab's Bio-energy group. The robot runs on electricity extracted from biomass by 48 microbial fuel cells (the two concentric brick coloured rings). The robot is 90% 3D printed, and the plastic is recyclable.

 

 

 

 

 

I would love to see, in the near term, not only a new standard on Sustainable Robotics as a guide (and spur) for manufacturers, but the emergence of Sustainable Robotics as a thriving new sub-discipline in robotics.

New IET online course on Robot Ethics goes live

Big day today. My online course on Robot Ethics has been launched on the Institution of Engineering and Technology (IET) Academy web pages. The aim of the course is to give a comprehensive introduction to robot ethics and responsible robotics, and machine ethics. As well as ethical principles the course introduces powerful practical tools including Ethically Aligned Design (also called values driven design), emerging new ethical standards including BS8611 and the powerful method Ethical Risk Assessment,  IEEE 7001 on Transparency, and equally essential Ethical Governance, while showing how ethics, standards and regulation are linked. The course took the best part of 18 months to write, not least because of the strict formatting and style required for IET online courses. For academics, writing courses normally means just creating slides, but - to my surprise - IET online courses are narrated by professional voice actors, so I had to write the narration for each slide. Plus, alot of tests to help students to self-test their understanding.

The course is organized as 10 one hour units, each with several modules, and tests at the end of each module and at the end of the unit. Here is the outline syllabus.

Unit 1: What is Robot Ethics?

This unit defines what we mean by an intelligent robot, robot ethics and ethical robots.

Module 1: Defines what we mean by a robot and robot autonomy, while explaining the difference between first wave (i.e. industrial) robots and second wave (i.e. social) robots
Module 2: Defines intelligence and clarifies the distinction between robotics and Artificial Intelligence (AI)
Module 3: Robot/AI ethics: ethics for humans and responsible robotics
Module 4: Machine ethics: ethics for robots

Unit 2: Inspired by Asimov – The EPSRC Principles of Robotics

This unit focuses on the influential EPSRC Principles of Robots.

Module 1: Asimov’s Three Laws of Robotics, their limitations, and their contribution to robot ethics
Module 2: Why robot ethics are so important today
Module 3: The EPSRC Principles of Robotics
Module 4: Responsible Robotics

Unit 3: An Overview of Ethical Frameworks for AI

This unit looks at some of the more recent ethical frameworks proposed for robotics and AI.

Module 1: A Proliferation of Principles. A helicopter view of all of the ethical frameworks for robotics and artificial intelligence published since Asimov’s laws of robotics. Including what an ethical framework is and what it does and does not offer.
Module 2: The Future of Life Institute Asilomar principles for beneficial AI
Module 3: The UNI Global Union Top 10 Principles for Ethical AI
Module 4: The European Commission’s High Level Expert Group on AI Ethics Guidelines for Trustworthy AI

Module 5: The OECD Principles of AI

Module 6: Summary: comparing ethical frameworks and their limitations 

Unit 4: Ethical Standards in Robotics

This unit explores emerging ethical standards.

Module 1: Standards, an introduction
Module 2: An Ethical Standard - British Standard BS8611:2016 A Guide to the Ethical Design of Robots and Robotic Systems
Module 3: Ethical Risk Assessment based on BS8611, including a Case Study
Module 4: Standards in Practice

Unit 5: Ethically Aligned Design in Robotics and AI

This unit introduces the IEEE global ethics initiative and ethically aligned design.

Module 1: The IEEE Global Ethics Initiative
Module 2: The IEEE General Principles
Module 3: Ethically Aligned Design
Module 4: The P70XX Human Standards

Unit 6: Transparency and Explainability in Robotics and AI

This Unit explores transparency, and the related topic of accident investigation.

Module 1: Introduction to Transparency and Explainability
Module 2: The IEEE P7001 Standard on Transparency in Autonomous Systems
Module 3: Robot Accident Investigation, an introduction

Unit 7: Ethical Governance for Robotics

This unit focuses on ethical governance for robotics.

Module 1: How do we trust our technology?
Module 2: A Roboethics Roadmap, linking ethics, standards and regulation
Module 3: Robotics Law and Regulation, with examples from Drones, Autonomous Vehicles and Assisted Living robots
Module 4: A framework for ethical governance

Unit 8: Machine Ethics 1 – An Asimovian Ethical Robot

In this unit, we will explore machine ethics, and ask the question: is it possible to build a moral machine?

Module 1: A thought experiment: is it possible to build a moral machine?
Module 2: The Consequence Engine
Module 3: Experimental trials of an Asimovian ethical robot

Unit 9: Machine Ethics 2 – Approaches, Risks and Governance

Module 1: Categories of ethical agency
Module 2: Approaches to building ethical robots
Module 3: The risks of ethical robots
Module 4: The governance of ethical machines

Unit 10: Final Assessment