Archive for the ‘Peer review’ Category

RealClimate libel threat

February 22, 2011

The publishers of the journal Energy & Environment have threatened libel action against the RealClimate blog because of remarks they made about the peer review process in E&E.

The key quote that upset them, from this post, was:

“The evidence for this is in precisely what happens in venues like E&E that have effectively dispensed with substantive peer review for any papers that follow the editor’s political line.”

I think it is somewhat cowardly of E&E to threaten this libel action as it is often used to silence critics when their arguments can’t be refuted with evidence (see the RealClimate post for examples and then go and sign the petition to reform English libel laws).

If E&E can defend their peer review system, then I think should do so. What would be even better is if E&E took the plunge and opened up their peer review process and became a model for other journals. After all, many of the “skeptics” who have published in or are on the editorial board of E&E have called for the peer review process to open up. Here’s a great opportunity to make some progress on that front!

I wrote their publishers a letter explaining my view:

Dear Bill Hughes,

I have just read the letter that you sent to RealClimate regarding their criticism of the peer review process in E&E. I am disappointed that you consider a libel threat a valid response to their post.

Indeed, if E&E has a robust peer review process then this opportunity could be taken to open that process up. In doing so, you could not only refute the claims made by RealClimate but also become a trailblazer for a system to replace the current model of anonymous/closed peer review, which is often criticised for favouring the consensus view on subjects like climate change.

I look forward to an imaginative and constructive response from E&E.

Yours sincerely,

Andrew Russell

Advertisements

Just a bit of fun?

December 16, 2010

Environmental Microbiology recently published a list of anonymous referee quotes from their reviews from the preceeding year. They did this in the interest of seasonal merriment.

Some of the quotes are quite funny:

“This paper is desperate. Please reject it completely and then block the author’s email ID so they can’t use the online system in future.”

“Reject – More holes than my grandad’s string vest!”

I’m not so sure that publishing this stuff is a good idea. Here are a few reasons:

1) The intro to the list says “Here are some quotes from reviews made over the past year”. To me, this implies that they were sent to the authors of the submitted papers. But they actually read much more like Comments to the Editor, so the authors of the papers in question wouldn’t see these comments. This list has got a bit of media attention and I suspect that to people that haven’t been involved in the review process and don’t understand the context will assume that scientists are rude, tactless and take advantage of the anonymity that peer review allows.

2) The other problem with publishing Comments to the Editor is that the reviewers probably wrote them without thinking that they would be published – they are often explicitly listed as confidential on the reviewer comments form (I don’t know if this is the case for EM). I hope the EM editor asked permission before publsihing these comments. I know that not all editors are so considerate: an editor of a journal I once wrote a review for cut and pasted some of my “confidential” comments into my review for the authors. There was nothing offensive in those comments but it was information that I thought would be useful to the editor that I didn’t think would help the authors revise the paper. If I’d have known it would be sent to the authors I would have written it in a different style.

3) Perhaps more worrying is if reviewers for EM are writing their reviews in the knowledge that the editor may use their “funny” comments in the December issue (this is the 2nd year that EM have done this list). As an Associate Editor of a journal, I often find that the comments written specifically for me are the most useful information in deciding the outcome of the review process. If reviewers are trying to get a punchline in that statement then that runs the risk of making the review process harder than it needs to be or skewing the reviewer’s tone.

4) I don’t know how many manuscripts EM have submitted each year but the authors of the paper that this refers to could probably work it out:

“The presentation is of a standard that I would reject from an undergraduate student. Take Table 1: none of the data has units or an explanation. Negative controls gave a positive signal, but there is no explanation of why and how this was dealt with; just that it was different.”

It’s not very polite (or funny actually).

5) Finally, a few of the comments give the impression that the peer-review process is a bit cliquey:

“Ken, I would suggest that EM is setting up a fund that pays for the red wine reviewers may need to digest manuscripts like this one. (Ed.: this excellent suggestion was duly proposed to the Publisher. However, given the logistical difficulties of problem-solving within narrow time frames, combined with the known deleterious effect of transport on good wine, a modification of the remedy was adopted, namely that Editors would act as proxies for reviewers with said digestive complaints.)”

This kind of comment makes me think that having open, non-anonymous peer-review would be for the best.

Anyway, I suspect I’m being a bit of a killjoy but I’m not sure the laughs are worth the potential damage that these quotes can do.