GoKawiilCredit: Javier Zayas/Getty
Papers with publicly available peer-review reports are less likely to be retracted than are those without public reports, a study has found. The authors say this could, in some cases, reflect greater confidence among researchers who choose to make the review process transparent.
The analysis, posted on preprint server MetaArXiv earlier this year1, examined 116,359 papers that have been published since May 2019 in open-access journals published by Public Library of Science (PLOS), based in San Francisco, California. PLOS journals give authors the choice to publish peer-review reports alongside their paper once the manuscript is accepted. The authors of 46,920 of the analysed papers — or slightly more than 40% — opted in.
One hundred and five of the papers with open referee reports were later retracted, compared with 321 of the papers for which the peer-review process remained closed.
The study does not attempt to explain the association between open peer review and lower retraction rates, but in unpublished follow-up work described to Nature Index, study co-author Matt Spick, a research-integrity specialist at the University of Surrey in Guildford, UK, explored whether reviewer sentiment might offer clues.
One possibility was that PLOS authors, who choose whether to publish reviewer reports only after acceptance, might be more likely to opt in when reviews are favourable. To test this, Spick compared reviewer sentiments in PLOS journals with publicly available reports for papers in journals published by BMC, based in London. (The BMC portfolio is owned by Springer Nature, which publishes Nature.) He found no difference between the two data sets, suggesting that the lower retraction rate is unlikely to be explained by authors selectively publishing more-positive reviews.
Instead, Spick and Adrian Barnett, a statistician at the Queensland University of Technology in Brisbane, Australia, and co-author of the MetaArXiv preprint, hypothesize that researchers who are confident in their work and who broadly embrace open-science practices might be more likely to choose open peer review. As a result, the decision to opt for open peer review could itself serve as an indicator of article quality.
Jessica Polka, who develops biology teaching materials at MIT Open Learning, a research and education initiative in Boston, Massachusetts, agrees with this interpretation, saying that opting in to making referee reports public could be “a very useful signal to the reader that the authors are confident in the findings”.
Open or closed?
There are many variables at play that make it difficult to explain the results of the study one way or the other.
... continue reading