Peer Review

As a science student, you have no doubt learned that the “gold standard” of scientific evidence is empirical research published in peer-reviewed journals. Peer review is a hallmark of modern scientific practice and credibility, whereby scientists review each others’ work. This is typically done in a “blinded” fashion, whereby the authors of a paper, at least, do not know who the reviewers of the paper are (sometimes reviews are double-blinded, such that reviewers do not see the names or affiliations of the authors). As well, reviewers must declare that they have no conflict of interest in reviewing the work, such as a personal or professional relationship with the authors, or a financial incentive to recommend or prevent publication of the work. Peer reviewers are also not paid to perform this duty, but the vast majority of scientists engage in peer review for journals through a sense of collective duty; firstly, these scientists rely on their peers to review their own work, and secondly, peer review allows scientists to maintain quality control over their discipline. An added bonus is that reviewers get to see the latest scientific developments before others do.

While the principles of the peer review process are sound, they are not without problems. Firstly, the process typically involves an editor for the journal receiving a manuscript, and sending it out for peer review. Many papers are only reviewed by one or two peers, though this may go up to 5 or 6, but rarely more. Thus the people who review a paper may not be truly representative of the views of the wider scientific community. Furthermore, since peer review is unpaid labour, editors must rely on the good will — and availability — of reviewers. Editors also often do not know the people they invite as reviewers well, if at all, and so rely on a general principles of honesty and good faith (e.g., that a reviewer has appropriate expertise, and reviews with a neutral perspective). This process is thus prone to selection biases, whereby the people who provide reviews may neither be ideally-suited for the job, nor entirely neutral. Indeed, there are numerous cases of papers being rejected by reviewers who clearly were incentivized to do so by a desire to publish similar work first, or even steal ideas and use them as their own.

These issues have led to the development of open peer review. In this process, the names of peer reviewers — and their reviews — are published along with the journal article itself. Typically the names of reviewers are only revealed to the authors, and the public, if the article is accepted for publication. Some scientists choose to “sign” their reviews with their names as a matter of principle. In some cases, controversially, reviewers have chosen to openly publish reviews of papers that they have recommended for rejection (see also this commentary).