Nature Ends Experiment With Open Peer Review, as Responses Are Disappointing, from the Chronicle
…while participation by authors was greater than anticipated, the posted remarks were disappointing. … Nearly half of the papers received no comments at all. Most of the messages made reasonable but minor points. Others, such as “nice work,” were far from useful.
Once a decision had been made to accept or reject a paper, the entire post was removed. “Any comments that were made on the paper before publication would be lost,” he said. “If you’re going to take time to think about and analyze a paper, you want what you say to be in a more permanent record.”
Actually, I entirely disagree with this last point. Blind reviews of papers are not in the permanent record either: there is no reason for the author, the reviewer, or the publisher to archive them, and I doubt that any do. Though actually publishers may, given that much of the publishing lifecycle is managed with some form of content management app, so reviews may be stored; I don’t know. Nevertheless, my understanding of the purpose of open review is that it allows more people to review papers than would do so in a blind review process, not that it allows reviews to be archived.
The Chronicle article links to a report on Nature’s site.
…we believe that this process works as well as any system of peer review can. Furthermore, in our occasional surveys of authors we receive strong signals of satisfaction: in the most recent survey, 74% agreed with the statement that their paper had been improved by the process, 20% felt neutral, while 6% disagreed.
We sent out a total of 1,369 papers for review during the trial period. The authors of 71 (or 5%) of these agreed to their papers being displayed for open comment. Of the displayed papers, 33 received no comments, while 38 (54%) received a total of 92 technical comments.