Wednesday, September 08, 2010

Darn - conference paper soundly rejected

As someone who believes in - and from time-to-time advocates - the Open Science approach, I need to practise what I preach. That means being open about the things that don't go according to plan in a research project - including when papers that you think are really great get rejected following peer review. So, let me 'fess up. A paper I submitted to the highly regarded conference Distributed Autonomous Robotic Systems, describing results from the Artificial Culture project, has just been soundly rejected by the reviewers.

Of course, having papers rejected is not unusual. And, like most academics, I tend to react with indignation ("how dare they"), dismissal ("the reviewers clearly didn't understand the work") and embarrassment (hangs head in shame). After a day or two the first two feelings subside, but the embarrassment remains. None of us likes it when our essays come back marked C-. That is why this blog post is not especially comfortable to write.

My paper had four anonymous reviews, and each one was thorough and thoughtful. And - although not all reviewers recommended rejection - the overall verdict to reject was, in truth, fully justified. The paper, titled A Multi-robot Laboratory for Experiments in Embodied Memetic Evolution failed to either fully describe the laboratory, or the experiments. Like most conference papers there was a page limit (12 pages) and I tried to fit too much into the paper.

So, what next for this paper? Well the work will not be wasted. We shall revise the paper - taking account of the reviewers comments - and submit it elsewhere. So, despite my embarrassment, I am grateful to those reviews (I don't know who you are but if you should read this blog - thank you!).

And for Open Science. Well, a fully paid-up card carrying Open Scientist would publish here the original paper and the reviews. But it seems to me improper to publish the reviews without first getting the reviewers' permission - and I can't do that because I don't know who they are. And I shouldn't post the paper either, since to do so would compromise our ability to submit the same work (following revision) somewhere else. So Open Science, even with the best of intentions, has its hands tied by publications protocols.

3 comments:

  1. This is a classic Prisoner's Dilemma. If you're the only person to own up, you lose; if we all own up, we'll all win.

    ReplyDelete
  2. This post is a good lesson to junior researchers because many of them keep in mind that it is easy to get a paper accepted in any conference/journal if there is a "big name"- a well-known person in a scientific community"- on the paper.
    Your case is a good evidence showing that such thoughts are not true. I think the message " In science, everyone is equal" should be sent to all junior researchers. The belief encourages them to stand on their legs, instead under the shadow of someone else.

    On the one hand, I know you might take time to digest this result. On the other hand, I am sure that you receive more respect from people who have known you, your works, ever particpited in your lectures, and read your blog by this post.

    ReplyDelete
  3. Thanks AAUswarm - your 'good lesson' observation is very interesting and something that hadn't occurred to me. You're right - in science all work should be equally judged on its merit only, regardless of the names attached to it.

    ReplyDelete