AI and why we should all be worried - AI's dirty secrets

I gave a talk on AI for the Swindon Science Cafe. Here below are the slides, followed by my notes.

Slide 1

Hi, my name is Alan Winfield. Thank you Rod and Claire for inviting me to speak this evening.

Slide 2

So, what does a robot and AI ethicist do? Well, the short answer is worry.

I worry about the ethical and societal impact of AI on individuals, society and the environment. Here are some keywords on ethics, reflecting that we must work toward AI that respects Human Rights, diversity and dignity, is unbiased and sustainable, transparent, accountable and socially responsible. I also work in Standards with both the British Standards Institute and the International IEEE Standards Association.

Slide 3

Before getting into the ethics of AI I need to give you a quick tutorial on machine learning.

The most powerful and exciting AI today is based on Artificial Neural Networks (ANNs). Here is a simplified diagram of a Deep Learning network for recognizing images. Each small circle is a *very* simplified mathematical model of biological neurons, and the outputs of each layer of artificial neurons feed the inputs of the next layer. In order to be able to recognise images the network must first be trained with images that are already labelled - in this case my dear late dog Lola.

But in order to reliably recognise Lola the network needs to be trained not with one picture of Lola but many. This set of images is called the training data set and without a good data set the network will not work at all or will be biased. (In reality there will need to be not 4 but hundreds of images of Lola).

But even a simple ANN can get things wrong. A famous example was an ANN like this trained on pictures of wolves. After training they input of a bear, but the network identified it as a wolf. Why? Because all of the wolf pictures had snowy backgrounds, so the network learned to recognize snow, not a wolf. The bear also had a snowy background.

This is one example of what we now call a ‘hallucination’ in big AIs.

Slide 4

We’ve been worrying about the existential threat of AI for a long time: here is a piece I wrote for the Guardian in 2014, when ‘the singularity was the main thing we worried about.

The singularity is the idea that as soon as AI is smarter than humans then the AIs will rapidly improve themselves, with unforeseeable consequences for human civilization.

But the singularity is a thing for the techno-utopians: wealthy middle-aged men who regard the singularity as their best chance of immortality. They are Singularitarians, some of whom appear prepared to go to extremes to stay alive for long enough to benefit from a benevolent super-AI - a manmade god that grants transcendence.

And it's a Thing for the doomsayers, the techno-dystopians. Apocalypsarians who are equally convinced that a superintelligent AI will have no interest in curing cancer or old age, or ending poverty, but will instead - malevolently or maybe just accidentally - bring about the end of human civilisation as we know it. History and Hollywood are on their side. From the Golem to Frankenstein's monster, Skynet and the Matrix, we are fascinated by the old story: man plays god and then things go horribly wrong.

Slide 5

Today we have influential scientists who worry about Artificial General Intelligence (AGI). Notable among these is physicist and cosmologist Max Tegmark.

At the AI safety summit in February 2025 Tegmark argued that we need a middle pathway between No AI and Uncontrollable AGI, which he calls guaranteed safe tool AI. He suggested a policy solution in which the US and China both launch national safety standards preventing their own AI companies from building AGI. Leading to what Tegmark rather optimistically calls ‘an age of unprecendented global prosperity powered by safe tool AI.

Is there an existential threat from AI itself? No. I fear human stupidity much more than artificial intelligence.

So should we be worried? Yes. But the things I worry about are rather more down to earth.

In the rest of this talk I will consider the energy costs of AI, then the human costs.

Slide 6

In 2016 Go champion Lee Sedol was famously defeated by DeepMind's AlphaGo. It was a remarkable achievement for AI. But consider the energy cost. In a single two-hour match Sedol burned around 170 kcals: roughly the amount of energy you would get from an egg sandwich. Or about 1 Watt – the power of an LED night light.

In the same two hours the AlphaGo machine reportedly consumed about 50,000 Watts. The same as a 50 kW generator for industrial lighting. And that's not taking account of the energy used to train AlphaGo.

Slide 7

A paper published in 2019 paper revealed, for the first time, estimates of the carbon cost of training large AI models for natural language processing such as machine translation. The carbon cost of simple models is quite modest, but with tuning and experimentation the carbon cost leaps to 7 times the carbon footprint of an average human in one year (or 2 times if you're an American).

And the energy cost of optimizing the biggest model is a staggering 5 times the carbon cost of a car over its whole lifetime, including manufacturing it in the first place. The dollar cost of that amount of energy is estimated at between one and 3 million US$. (Something that only companies with very deep pockets can afford.)

These energy costs seem completely at odds with the urgent need to meet sustainable development goals. At the very least AI companies need to be honest about the huge energy costs of machine learning.

Slide 8

At the same Paris meeting earlier this year AI ethicist Kate Crawford drew attention to both energy and water costs of AI. Crawford predicts that the energy cost of training generative AIs will soon overtake the total energy consumption of industrialized nations such as Japan.

She also drew attention to the colossal amounts of clean water that AI server farms need to keep them cool. Water that is already a scarce resource.

One study estimated that ChatGPT-3 requires 700,000 litres of clean water for training, And that each user interaction costs around half a litre of water.

Source: https://interestingengineering.com/innovation/training-chatgpt-consumes-water

Slide 9

The very same Kate Crawford, together with a colleague, produced this extraordinary map of the entire process behind the Amazon Echo.

The remarkable Anatomy of an AI System, shows The Amazon Echo as an anatomical map of human labour, data and planetary resources. 

The map is far too detailed to see on this slide but let me just zoom in on the top of this very large iceberg where we find the amazon echo and its human user, shown here in a yellow dashed box.

Slide 10

At the very top of this pyramid of materials and energy (on the left) and waste (on the right) is you – the user of the Amazon Echo – and your unpaid human labour providing habits and preferences that will be used as training data. 

I strongly recommend you check this out. It is truly eye opening.

Slide 11

Now I want to turn to the human cost of AI.

It is often said that one of the biggest fears around AI is the loss of jobs. In fact the opposite is happening. Many new jobs are being created, but the tragedy is that they are not great jobs, to say the least. Let me introduce you to 3 of these new kinds of jobs.

Conversational AI or chat bots also need human help. Amazon for instance employs thousands of both full-time employees and contract workers to listen to and annotate speech. The tagged speech is then fed back to Alexa to improve its comprehension. In 2019 the Guardian reported that Google employs around 100,000 temps, vendors and contractors: literally an army of linguists to create the handcrafted data sets required for Google translate to learn dozens of languages. Not surprisingly there is a considerable disparity between the wages and working conditions of these workers and Google's full-time employees.

AI tagging jobs are dull, repetitive and in the case of the linguists highly skilled.

Slide 12

Consider AI tagging of images. This is the manual labelling of objects in images to, for instance, generate training data sets for driverless car AIs. Better (and safer) AI needs huge training data sets and a whole new outsourced industry has sprung up all over the world to meet this need. Here is an AI tagging factory in China.

Slide 13

But by far the worst kind of new white-collar job in the AI industry is content moderation.

These tens of thousands of people, employed by third-party contractors, are required to watch and vet offensive content: hate speech, violent pornography, cruelty and sometimes murder of both animals and humans for Facebook, YouTube and other media platforms. These jobs are not just dull and repetitive they are positively dangerous. Harrowing reports tell of PTSD-like trauma symptoms, panic attacks and burn out after one year, alongside micromanagement, poor working conditions and ineffective counselling. And very poor pay - typically $28,800 a year. Compare this with average annual salaries at Facebook of $250,000+.

Slide 14

The extent to which AI has a human supply chain was a big revelation, and I am an AI insider! The genius designers of this amazing tech rely on both huge amounts of energy and a hidden army of what Mary Gray and Siddhartha Suri call Ghost Workers.

I would ask you to consider the question: how can we, as ethical consumers, justify continuing to make use of unsustainable and unethical AI technologies?

Slide 15  

AI ethics are important because AIs are already causing harm. Actually a *very* wide range of harms.

Fortunately, there is an excellent crowdsourced database which collects reports of accidents and near misses involving robots (including autonomous vehicles) and AIs.

The AI incidents database covers both accidents and near misses. I strongly recommend you check it out.

It is important to note that because the AI incidents database is crowdsourced from accidents and near misses that made it into the press and media, what we see is almost certainly only the tip of the iceberg of the harms being done by AI.

The database contains some truly shocking cases. One is a 14-year-old boy, who died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. Source: Can A.I. Be Blamed for a Teen’s Suicide? New York Times, Oct 2024.

The database also highlights the criminal use of AI. Examples include criminals phoning parents claiming they have kidnapped a child and demanding a ransom, with deepfake audio of the teenage girl audible in the background. And there are many examples of sextortion, using deepfake AI generated video of famous individuals engaged in sex acts.

The database really highlights the sad truth that AI is a gift to criminals.

Slide 16  

Another more recent database tracks the misuse of AI by lawyers when preparing court cases. The database only shows those instances where the judge (or another officer in the court) spotted the hallucinated decisions, citations or quotations. The cases were thrown out, and in some cases the lawyers found to be using Ai were fined or reported to their bar associations.

While this is not criminal misuse of AI, it does demonstrate a lack of understanding of AI, or at best, naivety. Perhaps the real culprits are hard pressed paralegals. This database underlines the need for professionals to be property trained in AI and its weaknesses.

Since I grabbed this screen shot the number if cases reported has grown.

Slide 17

Lawyer Graziano Mioli elegantly argues that we have a categorical imperative to be imperative when interacting with AIs. Noting that this is not an invitation to be rude.

For Graziano's slides see https://www.youtube.com/watch?v=tjBnGN4u1GA&ab_channel=GrazianoMioli  

I can see why a majority of people interact kindly with AIs. I think it reflects well on those who do say ‘please’, for the Kantian reason that we should not get into the habit of acting unkindly. (Kant famously argued that you should not kick a dog - not for animal welfare reasons, but because you might then be more inclined to kick a human).

See https://www.theaihunter.com/news/ai-etiquette-why-some-people-say-please-to-chatbots/

Slide 18

Having mostly elaborated on the dangers of AI, I want to finish on a positive.

We already enjoy the benefit of useful and reliable AI technology, like smart maps and machine translation. DeepMind's diagnostic AI can detect over 50 eye diseases from retinal scans as accurately as a doctor, and DeepScribe provides automated note taking during a consultation, linking with patient electronic health records

Thank you! 

 

This talk is an extended and updated version of a short talk I gave in June 2019.

Open science: preaching what I practice

I was very pleased to be invited to Science, Innovation and Society: achieving Responsible Research and Innovation last week. I was asked to speak on open science - a great opportunity to preach what I practice. Or at least try to practice. Doing good science research is hard, but making that work open imposes an extra layer of work. Open science isn't one thing - it is a set of practices which range from making sure your papers are openly accessible, which is relatively easy, to open notebook science, which makes the process open, not just the results, and is pretty demanding. In my short introduction during the open science panel I suggested three levels of open science. Here are those slides:



In my view we should all be practising level 0 open science - but don't underestimate the challenge of even this minimal set of practices; making data sets and source code, etc, available, with the aim of enabling our work to be reproducible, is not straightforward.

Level 0 open science is all one way, from your lab to the world. Level 1 introduces public engagement via blogging and social media, and the potential for feedback and two-way dialogue. Again this is challenging, both because of the time cost and the scary - if you're not used to it - prospect of inviting all kinds of questions and comments about your work.  In my experience the effort is totally worthwhile - those questions often make me really think, and in ways that questions from other researchers working in the same field do not.

Level 2 builds on levels 0 and 1 by adding open notebook science. This takes real courage because it opens up the process, complete with all the failures as well as successes, the bad ideas as well as the good; open notebook science exposes science for what it really is - a messy non-linear process full of uncertainty and doubts, with lots of blind alleys and very human dramas within the team. Have I done open notebook science? No. I've considered it for recent projects, but ruled it out because we didn't have the time and resources or, if I'm honest, team members who were fully persuaded that it was a good idea.

Open science comes at a cost. It slows down projects. But I think that is a good, even necessary, thing. We should be building those costs into our project budgets and work programmes, and if that means increasing the budget by 25% then so be it. After all, what is the alternative? Closed science..? Closed science is irresponsible science.

---

At the end of the conference the Rome Declaration on Responsible Research and Innovation was published.

Robotics needs to get Political

A couple of weeks ago I was a panelist on a public debate at the 2014 Battle of Ideas. The title of the debate was The robots are coming: friends or foes? with a focus not on the technology but the social and economic implications of robotics. One of the questions my brilliant fellow panelists and I were asked to consider was: Will the ‘second machine age’ bring forth a new era of potential liberation from menial toil or will the short-term costs for low-paid workers outstrip the benefits?

Each panelist made an opening statement. Here is mine:

Most roboticists are driven by high ideals. 

They, we, are motivated by a firm belief that our robots will benefit society. Working on surgical robots, search and rescue robots, robots for assisted living or robots that can generate electricity from waste, my colleagues in the Bristol Robotics Lab want to change the world for the better. The lab's start up companies are equally altruistic: one is developing low cost robotic prosthetic hands for amputees, three others are developing materials, including low cost robots, for education

Whatever their politics, these good men and women would I suspect be horrified by the idea that their robots might, in the end, serve to further enrich the 0.1%, rather than extend the reach of robotics to the neediest in society.

I was once an idealist - convinced that brilliant inventions would change society for the better just by virtue of being brilliant.

I'm older now. 

For the last 5 years or so I have become an advocate for robot ethics. 

But in the real world, ethics need teeth. In other words we need to move from ethical principles, to standards, to legislation.

So I’m very pleased to tell you that in the last few days the British Standards Institute working group on robot ethics has published - for comments - a proposed new Guide to the ethical design and application of robots and robotic systems.

In the draft Guide we have identified ethical hazards associated with the use of robots, and suggest guidance to either eliminate or mitigate the risks associated with these ethical hazards. We outline 15 high level ethical hazards under four headings: societal, use, legal/financial and environmental.

Like any transformative technology robotics holds both promise and peril. As a society we need to understand, debate, and reach an informed consensus about what robots should do for us, and even more importantly, should not do. 

Ladies and Gentlemen: Robotics, I believe, needs to get political.

The debate was recorded and is on soundcloud here:




It was a terrific debate. We had a very engaged audience with hugely interesting - and some very challenging - questions. For me it was an opportunity to express and discuss some worries I've had for awhile about who will ultimately benefit from robotics. In summing up toward the end I said this:

Robotics has the potential for huge benefit to society but is too important to leave to free-market capitalism.

Something I believe very strongly.

Swarm robotics at the Science Museum

Just spent an awesomely busy weekend at the Science Museum, demonstrating Swarm Robotics. We were here as part of the Robotville exhibition, and - on the wider stage - European Robotics Week. I say we because it was a team effort, led by my PhD student Paul O'Dowd who heroically manned the exhibit all four days, and supported also by postdoc Dr Wenguo Liu. Here is a gallery of pictures from Robotville on the science museum blog, and some more pictures here (photos by Patu Tifinger):

Although exhausting, it was at the same time uplifting. We had a crowd of very interested families and children the whole time - in fact the organisers tell me that Robotville had just short of 8000 visitors over the 4 days of the exhibition. What was really nice was that the whole exhibition was hands-on, and our sturdy e-puck robots - at pretty much eye-level for 5-year olds, attracted lots of small hands interacting with the swarm. A bit like putting your hand into an ants nest (although I doubt the kids would have been so keen on that.)

Let me explain what the robots were doing. Paul had programmed two different demonstrations, one with fixed behaviours and the other with learning.

For the fixed behaviour demo the e-puck robots were programmed with the following low-level behaviours:

1. Short-range avoidance. If a robot gets too close to another robot or an obstacle then it turns away to avoid it.
2. Longer-range attraction. If a robot can sense other robots nearby but gets too far from the flock, then it turns back toward the flock. And while in a flock, move slowly.
3. If a robot loses the flock then it speeds up and wanders at random in an effort to regain the flock (i.e. another robot).
4. While in a flock, each robot will communicate (via infra-red) its estimate of the position of an external light source to nearby robots in the flock. While communicating the robot flashes its green body LED.
5. Also while in a flock, each robot will turn toward the 'consensus' direction of the external light source.

The net effect of these low-level behaviours is that the robots will both stay together as a swarm (or flock), and over time, move as a swarm toward the external light source. Both of these swarm-level behaviours are emergent because they result from the low-level robot-robot and robot-environment interactions. While the flocking behaviour is evident in just a few minutes, the overall swarm movement toward the external light source is less obvious. In reality even the flocking behaviour appears chaotic, with robots losing each other, and leaving the flock, or several mini-flocks forming. The reason for this is that all of the low-level behaviours make use of the e-puck robots' multi-purpose Infra-red sensors, and the environment is noisy; in other words because we don't have carefully controlled lighting there is lots of ambient IR light constantly confusing the robots.

The learning demo is a little more complex and makes use of an embedded evolutionary algorithm, actually running within the e-puck robots, so that - over time - the robots learn how to flock. This demo is based on Paul's experimental work, which I described in some detail in an earlier blog post, so I won't go into detail here. It's the robots with the yellow hats in the lower picture above. What's interesting to observe is that initially, the robots are hopeless - constantly crashing into each other or the arena walls, but noticeably over 30 minutes or so we can see the robots learn to control themselves, using information from their sensors. The weird thing here is that, every minute or so, each robot's control software is replaced by a great-great-grand child of itself. The robot's body is not evolving, but invisibly its controller is evolving, so that later generations of controller are more capable.

The magical moment of the two days was when one young lad - maybe 12 years old, who very clearly understood everything straight away and seemed to intuit things I hadn't explained - stayed nearly an hour explaining and demonstrating to other children. Priceless.

How to make a fool of yourself on national radio

Being interviewed live on national radio is an interesting experience.

It's not so bad when you're in a studio face to face with the interviewer. Then there's a proper sense of occasion, of being there for a purpose, something to rise to.

But being interviewed by telephone is an altogether different and more risky proposition. Why risky? Let me set the scene. You've agreed to be interviewed by a national radio station that has, hitherto, never blipped onto your cultural radar. The producer called and asked if you would be able to comment, in the science slot of the breakfast show, about a recent newspaper article listing the top 10 reasons that mankind could be wiped out this century. In particular the one that predicts mankind will, within 40 years, build super-intelligent robots who promptly (and ungratefully) enslave their creators. Quickly passing over your observation that said producer seems surprisingly laid back, you say to yourself - can't be so bad - they have a science slot. And of course you would be grateful for the opportunity to explain why this particular prediction is laughably absurd.

You rise early the following morning, after checking the news piece and giving some thought to how you can counter this particular piece of futurology. (Which turns out to be based on the mistaken assumption that because processing power is doubling roughly every 2 years, then robot intelligence is doing the same.)

With 20 minutes to spare you find the radio station on the Interweb and click the listen now button. The presenter starts to talk about robots-taking-over-the-world and invites a phone in. He wants listeners to phone with mad robot inventions and introduce them with a robot voice. Hmmm. At this point you begin to realise that the science slot doesn't have quite the level of gravitas that you might have hoped for.

Then the phone rings. Butterflies. Ok, normal. It's the laid back producer again. After a few minutes listening to the radio on the phone you hear yourself being introduced and you're on. This bit is always weird. You're on the phone with a few hundred thousand people on the other end. Just focus. It's only a conversation with some guy. Nevermind that he's called Xane. Or the fact that he just egregiously misquoted the article by inserting the words 'taking-over-the-world' between 'probability of super-intelligent robots' and 'high'.

The first couple of questions are kind of ok. More or less what you expected. You carefully explain that no, in your opinion it's extremely unlikely that we will build robots with super-human intelligence in the next 40 years and, even if we did, why should they be evil and take over the world (or more to the point why would we make them evil). Then some relatively innocuous questions: What is the most powerful robot in the world - is it Asimo? Er no, Asimo is actually remotely controlled by a team of 6. What about that freaky monkey robot with the robot arm? Well, that's not so much a robot as work to improve neural electronic interfaces to help people with smart prostheses.

Then just when you think it's all over you get the inevitable mad-question-at-the-end.

Q. But if robots did take over the world, what would we call them?

A. I really don't think robots are going to take over the world. 

Q. (More insistently this time) Yes, but if they did. What would we call them?

A. No, they really aren't going to take over the world.

Q. (Even more insistently) But what if they did? What would we call them?

Then you make a fool of yourself on the radio by wearily saying 'evil robot master' or somesuch nonsense, thus eliciting the triumphal response from Xane and his co-presenter: Aha! See, the professor says so. Robots really are going to take over the world.