Our first mock social robot accident and investigation

Robot accidents are inevitable. These days the likelihood of serious accidents involving industrial robots is pretty low (but not zero), because such robots are generally inside safety cages. But a newer generation of social robots - robots designed to interact directly with people, including vulnerable elderly people or children - means that accidents are now much more likely. And if we also take into account ethical harms alongside physical harms, then the potential for accidents increases still further. Psychological harms include addiction, over trusting, or deception, and societal harms include privacy violations. For more on these ethical harms see my blog post outlining an ethical risk assessment of a smart robot teddy bear.

It has puzzled me for some years that there has been almost no research on robot accident investigation. In the RoboTIPS project we are addressing this deficit by developing both the technology - which we call an Ethical Black Box (EBB) - and the processes of robot accident investigation. One of the most exciting aspects of RoboTIPS is that we're running a series of mock, i.e. staged, social robot accidents in order to road test the EBB and investigation processes in as close to a real situation as is feasible in a research project. RoboTIPS started in March 2019, but then just as we were ready to trial our first mock accident the Covid pandemic hit, and closed down the lab.

So it was great that last week we finally managed to run the a pilot of our first (of three) mock accident scenarios. The scenario, based around an assisted living robot helping an elderly person to live independently, was sketched out in late 2019, and then - during the lockdown - rehearsed in a number of online events, including a podcast radio play for Oxford Sparks and CSI Robot during the UKRAS Festival of Robotics 2021.

Here is the scenario:

Imagine that your elderly mother, or grandmother, has an assisted living robot to help her live independently at home. The robot is capable of fetching her drinks, reminding her to take her medicine and keeping in touch with family. Then one afternoon you get a call from a neighbour who has called round and sees your grandmother collapsed on the floor. When the paramedics arrive they find the robot wandering around apparently aimlessly. One of its functions is to call for help if your grandmother stops moving, but it seems that the robot failed to do this

To enact this scenario we needed a number of volunteers: one to act as Rose - the subject of the accident, a second as the neighbour who discovers the accident and raises the alarm, a third as the paramedic who attends to Rose, a fourth who acts in the role of the cleaner and a fifth in the role of manager of the group of homes in which Rose lives. We also needed volunteers to act as members of the accident investigation team who are called in to try and discover what happened, why it happened and, if possible, what changes need to be made to how to ensure the accident doesn't happen again.

This is the mock accident taking place in the kitchen of our assisted living studio. Left shows the neighbour, acted by Paul, discovering Ross, acted by Alex, injured on the floor. (Note the chair on its side.) Right is the paramedic, role-played by Luc, attending to Ross. Meanwhile the Pepper robot is moving around somewhat aimlessly.

Our brilliant Research Fellow Dr Anouk van Maris, who organised the whole setup, persuaded five colleagues from the Bristol Robotics Lab. All were male, so Rose became Ross. Only one volunteer: Alex, who played the part of Ross, was fully briefed. The other four role played brilliantly and, although they were briefed on their roles, they were not told what was going to happened to Ross, or the part the Pepper robot played (or maybe didn't play) in the accident. Two colleagues from Oxford, Lars and Keri, kindly volunteered to act as the accident investigators. Lars and Keri also had no prior knowledge of the circumstances of the accident, and had to rely on (i) inspecting the robot and the scene of the accident, (ii) the data from the robot's EBB, and (iii) testimonies from Ross, the neighbour, the paramedic, the cleaner and the facility manager.

Here we see Lars interviewing Medhi, who acted as the house manager, while Ben, acting as the cleaner, waits to be interviewed. Inside the studio Keri is interviewing the neighbour and parademic.

So, what were the findings of our accident investigators? They did very well indeed. Close examination of the EBB data, alongside consideration of the (not always reliable) witness testimony enabled Lars and Keri to correctly deduce the role that the robot played in the accident. They were also able to make several recommendations on operational changes.  But I will not reveal their findings in detail here as we intend to run the same mock accident again soon with a different set of volunteers and - in case any of them should read this blog - I don't want to give the game away!

Acknowledgements

Very special thanks to Dr Anouk van Maris. Also Dr Pericle Salvini, who worked with Anouk in finalising the detail of the scenario and during the pilot itself. Also, huge thanks to BRL volunteers Dr Alex Smith, Dr Paul Bremner, Dr Luc Wijnen, Mehdi Sobhani and Dr Ben Ward-Cherrier. And last but not least a very big thank you to Dr Lars Kunze, Oxford Robotics Institute and Keri Grieman, Dept of Computer Science, Oxford.

From the left: Pericle, Ben, Lars, Alex, Keri, Medhi, Paul, Anouk, Luc, Lola and me. Pepper is looking nervously at Lola.