As someone who believes in - and from time-to-time advocates - the Open Science approach, I need to practise what I preach. That means being open about the things that don't go according to plan in a research project - including when papers that you think are really great get rejected following peer review. So, let me 'fess up. A paper I submitted to the highly regarded conference Distributed Autonomous Robotic Systems, describing results from the Artificial Culture project, has just been soundly rejected by the reviewers.
Of course, having papers rejected is not unusual. And, like most academics, I tend to react with indignation ("how dare they"), dismissal ("the reviewers clearly didn't understand the work") and embarrassment (hangs head in shame). After a day or two the first two feelings subside, but the embarrassment remains. None of us likes it when our essays come back marked C-. That is why this blog post is not especially comfortable to write.
My paper had four anonymous reviews, and each one was thorough and thoughtful. And - although not all reviewers recommended rejection - the overall verdict to reject was, in truth, fully justified. The paper, titled A Multi-robot Laboratory for Experiments in Embodied Memetic Evolution failed to either fully describe the laboratory, or the experiments. Like most conference papers there was a page limit (12 pages) and I tried to fit too much into the paper.
So, what next for this paper? Well the work will not be wasted. We shall revise the paper - taking account of the reviewers comments - and submit it elsewhere. So, despite my embarrassment, I am grateful to those reviews (I don't know who you are but if you should read this blog - thank you!).
And for Open Science. Well, a fully paid-up card carrying Open Scientist would publish here the original paper and the reviews. But it seems to me improper to publish the reviews without first getting the reviewers' permission - and I can't do that because I don't know who they are. And I shouldn't post the paper either, since to do so would compromise our ability to submit the same work (following revision) somewhere else. So Open Science, even with the best of intentions, has its hands tied by publications protocols.