Open Science: from good intentions to hesitant reality

At the start of the Artificial Culture project we made a commitment to an Open Science approach. Actually translating those good intentions into reality has proven much more difficult than I had expected. But now we've made a start, and interestingly the open science part of this research project is turning into a project within a project.

So what's the story? Well, firstly we didn't really know what we meant by open science. We were, at the start, motivated by two factors. One, a strong sense that open science is a Good Thing. And, second, a rather more pragmatic idea that the project might be helped through having a pool of citizen scientists who would help us with interpretation of the results. We knew that we would generate a lot of data and also believed we would benefit from fresh eyes looking over that data, uncoloured - as we are - by the weight of hypotheses and high expectations. We thought we could achieve this simply by putting the whole project, live - as it happens - on the web.

Sounds simple: put the whole project on the web. And now that I put it like this, hopelessly naive. Especially given that we had not budgeted for the work this entails. So, this became a DIY activity fitted into spare moments using free Web tools, in particular Google Sites.

We started experimental work, in earnest, in March 2010 - about two and a half years into the project (building the robots and experimental infrastructure took about two years). Then, by July 2010 I started to give some thought to uploading the experimental data to the project web. But it took me until late October to actually make it happen. Why? Well it took a surprising amount of effort to figure out the best way of structuring and organising the experiments, and the data sets from those experiments, together with the structure of the web pages on which to present that data. But then even when I'd decided on these things I found myself curiously reluctant to actually upload the data sets. I'm still not sure why that was. It's not as if I was uploading anything important, like Wikileaks posts. Perhaps it's because I'm worried that someone will look at the data and declare that it's all trivial, or obvious. Now this may sound ridiculous but posting the data felt a bit like baring the soul. But maybe not so ridiculous given the emotional and intellectual investment I have in this project.

But, having crossed that hurdle, we've made a start. There are more data sets to be loaded (the easy part), and a good deal more narrative to be added (which takes a deal of effort). The narrative is of course critical because without it the data sets are just meaningless numbers. To be useful at all we need to explain (starting at the lowest level of detail):

1. what each of the data fields in each of the data files in each data set means;
2. the purpose of each experimental run: number of robots, initial conditions, algorithms, etc;
3. the overall context for the experiments, including the methodology and the hypotheses we are trying to test.

I said at the start of this blog post that the open science has become a project within a project and happily this aspect is now receiving the attention it deserves: yesterday project co-investigator Frances Griffiths spent the day in the lab here in Bristol, supported by Ann Grand (whose doctoral project is on the subject of Open Science and Public Engagement).

Will anyone be interested in looking inside our data, and - better still - will we realise our citizen science aspirations? Who knows. Would I be disappointed if no-one ever looks at the data? No, actually not. The openness of open science is its own virtue. And we will publish our findings confident that if anyone wants to look at the data behind the claims or conclusions in our papers they can.

---

Postscript: See also Frances Griffiths' blog post Open Science and the Artificial Culture Project

On optimal foraging, cod larvae and robot vacuum cleaners

On Monday I took part in a meeting of the Complex Systems Dynamics (CoSyDy) network in Warwick. The theme of the meeting was Movement in models of mathematical biology, and I heard amazing talks about (modelling) albatross flight patterns, e-coli locomotion, locust swarming and the spread of epidemics. (My contribution was about modelling an artificial system - a robot swarm.) Although a good deal of the maths was beyond me, I was struck by a common theme of our talks that I'll try and articulate in this blog post.

The best place to start is by (badly) paraphrasing a part of Jon Pitchford's brilliant description of optimal foraging strategies for cod larvae. Cod larvae, he explained, feed on patches of plankton. They are also very small and if the sea is turbulent the larvae have no chance of swimming in any given direction (i.e. toward a food patch), so the best course of action is to stop swimming and go where the currents take you. Of course the food patches also get washed around by the current so the odds are good that the food will come to you anyway. There's no point wasting energy chasing a food patch. Only if the sea is calm is it worthwhile for the cod larvae to swim toward a food patch. Thus, swim (toward food) when the sea is calm, but don't swim when it's rough, is the optimal foraging strategy for the cod larvae.

It occurred to me that there's possibly a direct parallel with robot vacuum cleaners, like the Roomba.  A robot vacuum cleaner is also foraging, not for food of course, but dirt in the carpet. For the robot vacuum cleaner the equivalent of a rough, turbulent, sea is a room with chaotically positioned furniture. The robot doesn't need a fancy strategy for covering the floor: it just drives ahead and every time it drives up to a wall or piece of furniture it stops to avoid a collision, makes a random turn and drives off again in a straight line. This is the robot's best strategy for reasonable coverage (and hence cleaning) of the floor in a chaotic environment (i.e. a normal room). Only if the room was relatively large and empty (i.e. a calm sea) would the robot (like the cod larvae) need a more sophisticated strategy for optimal cleaning - such as moving in a pattern across the whole area to try and find all the dirt.

Robot vacuum cleaners, like cod larvae, can exploit the chaos in their environment and hence get away with simple (i.e. stupid) foraging strategies. I can't help wondering - given the apparently unpredictable current economic environment - if there's really no point governments or individuals trying to invent sophisticated economic strategies. Perhaps the optimal response to economic turbulence is the KISS principle.

Why large robot swarms (and maybe also multi-cellular life) need immune systems.

Just gave our talk at DARS 2010, basically challenging the common assumption that swarm robot systems are highly scalable by default. In other words the assumption that if the system works with 10 robots, it will work just as well with 10,000. As I said this morning "sorry guys, that assumption is seriously incorrect. Swarms with as few as 100 robots will almost certainly not work unless we invent an active immune system for the swarm". The problem is that the likelihood that some robots partially fail - in other words fail in such a way as to actually hinder the overall swarm behaviour - quickly increases with swarm size. The only way to deal with this - and hence build large swarms - will be to invent a mechanism that enables good robots to both identify and disable partially failed robots. In other words an immune system.

Actually - and this is the thing I really want to write about here - I think this work hints toward an answer to the question "why do animals need immune systems?". I think it's hugely interesting that evolution had to invent immune systems very early in the history of multi-cellular life. I think the basic reason for this might be the very same reason - outlined above - that we can't scale up from small to huge (or even moderately large) robot swarms without something that looks very much like an immune system. Just like robots, cells can experience partial failures: not enough failure to die, but enough to behave badly - badly enough perhaps to be dangerous to neighbouring cells and the whole organism. If the likelihood of one cell failing in this bad way is constant, then it's self-evident that its much more likely that some will fail in this way in an organism with 10,000 cells than 10 cells. And with 10 million cells (still a small number for animals) it becomes a certainty.

Here is the poster version of our paper.