I’m thinking out loud here… Opinions expressed in this post may not reflect the opinions of the management, or indeed even of the author.

I recently reviewed 2 article manuscripts for 2 different journals. Journal A, in its instructions for authors, claims to be refereed but not double- or even single-blind; journal B does claim to be double-blind. The manuscript I received from journal A had the author’s name right on it. The MS I recieved from journal B did not have the author’s name on it and all identifying info had been removed as well as the editor could do. Still, dig ref is a small field & I think I have a pretty good idea about the type of work that everyone in it does. So I had my suspicions about who the authors were as I was reviewing the MS. Besides, a quick Google search would have confirmed it anyway: the title of the article is also the title of a conference presentation the authors will be giving in the near future. That said, I swear I did not Google the MS title, until after I finished the review.

So it occurred to me that even a double-blind review process is really only single-blind, despite the best efforts of the journal editor. And that doesn’t seem fair to me. I know, or at least suspect, who wrote the MS, but the author doesn’t know who reviewed it? That’s not fair. So this post is by way of me thinking aloud about what the consequences would be of reviewing being not-blind (sighted?). What if I, for example, signed my name to my reviews?

The article Reviewing a Manuscript for Publication did the rounds recently, & contains some good suggestions. If reviewing were sighted, would any of these change? Well, #9 is “Be kind.” If I had to sign my name to my reviews, I’d be politer. Not that I’m mean in my reviews — “this sucks! where’d you get your Ph.D., the University of Phoenix?” — nothing like that, of course. But I reviewed one MS once where by the end of reading it I still didn’t get what the purpose of it was, and I wrote that, or words to that effect. Would I have said that quite so baldly if I had to sign my name to the review? I’m not sure, maybe not. Other than that, I don’t see that any of the suggestions from that article would have to change in any significant way, if the reviewer were not writing anonymously.

Would there be any advantages to reviewing being sighted? Maybe. In all likelihood the author will already know the reviewers, at least by reputation. Academia is a small world, after all, & any given field smaller still. But if not, then this could be a good opportunity for the author to meet a new potential colleague, at least virtually: “Thanks for your recent reviews of my manuscript… I’d be interested to discuss some of your comments in more detail.” Or something like that, and then you get into a conversation. Potentially, anyway.

In a previous post, I linked to this article, Opening up BMJ peer review:

…for all new papers that we review, the BMJ will identify to authors the names of those who have reviewed their papers, including the names of our in house editorial and statistical advisers. But we expect to go further, researching as we go. Soon we will probably start to list reviewers at the end of articles. Then we may move to a system where authors and readers can watch the peer review system on the world wide web as it happens and contribute their comments. Peer review will become increasingly a scientific discourse rather than a summary judgment.

That editorial was written in 1999, though, & I don’t know what BMJ has done on this since. Another advantage of sighted reviewing that BMJ points out is the reviewing process providing formative rather than summative evaluation. I’ve always thought that was the intended purpose of reviewing anyway. So any means to make it more effective for that purpose would be good.

Would sighted review generate bad blood in academia? The BMJ piece suggests that

The main argument against open peer review — a sad one — is that junior reviewers will be reluctant to criticise the work of senior researchers for fear of reprisals. … By moving to open review we may thus be ruling out the best reviewers.

If a reviewer was rude in their review, I can see the author getting ticked off & trying to do some damage. But I can’t see a reasonable person getting angry over constructive criticism. That said, there are plenty of unreasonable people around. I have to ask myself, would I be willing to review a manuscript by a senior faculty member, and just to make the stakes higher, someone I was considering as an outside reviewer for my tenure case? Well, the benefit would be that it could serve as an entry point for a conversation with that person, which would be particularly a benefit if there wasn’t already a working relationship there. The downside is fear of reprisal. Would I take that risk, not knowing if the manuscript would be good? If I took the job, would I be perfectly honest in my review? Would I do a review if I thought I wouldn’t be perfectly honest? I have to admit, probably not.

I think I’ve talked myself around to the position where I started, that double-blind reviewing is a good idea. So the question is, how to fix the system so it actually is double-blind? Maybe I’ll write about that sometime, if I ever have any ideas on the subject.