Peer review has many aspects where bias is inevitable. Acknowledging this is the first step to managing it.
There is a platonic ideal of peer review in which reviewers are impartial and knowledgeable, allowing editors to make ‘correct’ decisions on the scientific studies or grant applications that have been submitted. In the real world this is impossible to achieve. The two requirements for a reviewer, in-depth knowledge and impartiality, stand in direct conflict with each other, connected by some loose analogue of Heisenberg’s uncertainty principle.
To produce a useful report a reviewer would ideally have intimate knowledge of the topic under review, gained by experience in the area. But with that kind of experience inevitably comes personally favoured theories and models so that any new data is seen through the prism of the reviewer’s previously established ideas. It is asking a lot of a human being to look as favourably on results that argue against their ideas as on those that confirm them. To be impartial a reviewer would have to be wholly naïve to the subject, and so have little value as a reviewer.
There is also the problem of professional relationships. Science is not a solitary pursuit. Scientists work in collaboration with others, as evidenced by the number of authors that appear on any of the articles in this or any other issue of Nature Plants; looking back, even the most eminent researcher was once a postgrad or postdoc in somebody else’s lab. There is a whole social network of relationships connecting one scientist with the next. In the field of mathematics this web has been mapped with the ‘Erdős number’, which records the number of steps of co-authorship that separate an individual from the highly collaborative Hungarian mathematician Paul Erdős. Those who have co-authored a paper with Erdős have an Erdős number of 1, those who have co-authored a paper with a co-author of Erdős have an Erdős number of 2, co-authors of co-authors of co-authors of Erdős have a score of 3, and so on. A similar numerical recreation has also been applied in the film industry, centred on the actor Kevin Bacon. Perhaps reviewers’ opinions should be weighted by a variant of this metric.
Many of the practices of peer review are attempts to minimise or compensate for the inevitable partiality of reviewers. The most obvious of these is the employment of multiple reviewers so that their expertise can be complementary and their preconceptions balanced. But even if we accept that no reviewer can be entirely impartial, that does not mean that no level of intellectual or personal connection between manuscript and reviewer can be tolerated. What then constitutes sufficient lack of impartiality to exclude a reviewer?
When we invite reviewers to advise us on a submitted study, the most common reason for a scientist to decline is a lack of time, but hard on its heels is that the potential reviewer feels that they have competing or conflicting interests. In the Nature Portfolio editorial policies, we define a competing interest as “financial and non-financial interests that could directly undermine, or be perceived to undermine the objectivity, integrity and value of a publication.” We also ask peer reviewers “to exclude themselves in cases where there is a significant conflict of interest, financial or otherwise.” However, there is a caveat: “Just as financial interests need not invalidate the conclusions of an article, nor do they automatically disqualify an individual from evaluating it.” That is all well and good, but where should the line be drawn? When is objectivity ‘undermined’, or a conflict of interest ‘significant’?
Some cases are clear. Scientists should not review the work of current collaborators. It is also not appropriate for a review to be in the same institution as the authors of a study. Reviewers and authors should not be recipients of any joint research grants. But what about past collaborations? How long is sufficient time since a co-authored paper for researchers no longer to be considered collaborators? How soon after a junior researcher has left a lab is it safe for their independent work to be assessed by their former boss?
There are no rules governing this. It is down to individual reviewers to decide for themselves whether they feel able to be sufficiently objective to review a paper, or possibly more importantly whether it would appear seemly to their research community. Of course, we editors are always keen to know about anything that might influence a reviewer’s advice, whether the reviewer considers it a serious conflict or not. We are also happy to advise whether we feel comfortable making a decision given a reviewer’s involvement in the subject matter of a piece of research or their connection to its authors. If in any doubt, it is usually best to decline.
It is becoming near standard practice to publicly acknowledge the reviewers of published papers. Nature Plants does this on individual papers if the reviewers agree to have their identities revealed, and anonymously if they do not. This sometimes displays what seem to be conflicts of interest, but which will most likely have been declared and considered when editorial decisions were being made. Also the path of review and revision, like that of true love, rarely runs smooth. It is very common that additional experiments are called for during peer review, and authorships can change as a consequence, introducing conflicts in revised submissions that were not present in the original. Yet all reviewers deserve thanks, whatever stage of the process they were involved in.
As we approach the end of the year we would like to thank all of our reviewers for all the invaluable advice that they have given us—and for their valiant efforts to overcome their inevitable biases while doing so.
About this article
Cite this article
Conflict resolution. Nat. Plants 7, 1435 (2021). https://doi.org/10.1038/s41477-021-01033-6