Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis.
BACKGROUND
The peer review process is a cornerstone of biomedical research. We aimed to evaluate the impact of interventions to improve the quality of peer review for biomedical publications.


METHODS
We performed a systematic review and meta-analysis. We searched CENTRAL, MEDLINE (PubMed), Embase, Cochrane Database of Systematic Reviews, and WHO ICTRP databases, for all randomized controlled trials (RCTs) evaluating the impact of interventions to improve the quality of peer review for biomedical publications.


RESULTS
We selected 22 reports of randomized controlled trials, for 25 comparisons evaluating training interventions (n = 5), the addition of a statistical peer reviewer (n = 2), use of a checklist (n = 2), open peer review (i.e., peer reviewers informed that their identity would be revealed; n = 7), blinded peer review (i.e., peer reviewers blinded to author names and affiliation; n = 6) and other interventions to increase the speed of the peer review process (n = 3). Results from only seven RCTs were published since 2004. As compared with the standard peer review process, training did not improve the quality of the peer review report and use of a checklist did not improve the quality of the final manuscript. Adding a statistical peer review improved the quality of the final manuscript (standardized mean difference (SMD), 0.58; 95 % CI, 0.19 to 0.98). Open peer review improved the quality of the peer review report (SMD, 0.14; 95 % CI, 0.05 to 0.24), did not affect the time peer reviewers spent on the peer review (mean difference, 0.18; 95 % CI, -0.06 to 0.43), and decreased the rate of rejection (odds ratio, 0.56; 95 % CI, 0.33 to 0.94). Blinded peer review did not affect the quality of the peer review report or rejection rate. Interventions to increase the speed of the peer review process were too heterogeneous to allow for pooling the results.


CONCLUSIONS
Despite the essential role of peer review, only a few interventions have been assessed in randomized controlled trials. Evidence-based peer review needs to be developed in biomedical journals.
Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA.|INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité - (CRESS), METHODS team, Paris, France.|Paris Descartes University, Paris, France.
Rachel
R
Bruce
INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité - (CRESS), METHODS team, Paris, France.|Paris Descartes University, Paris, France.|Emergency department, Hôpital Lariboisière, Assistance Publique des Hôpitaux de Paris, University Diderot, Paris, France.
Anthony
A
Chauvin
INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité - (CRESS), METHODS team, Paris, France.|Paris Descartes University, Paris, France.|Centre d'Épidémiologie Clinique, Hôpital Hôtel-Dieu Escalier A2, 1 er étage, Assistance Publique des Hôpitaux de Paris, 1 Place du Parvis Notre-Dame, Paris, 75004, France.
Ludovic
L
Trinquart
Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA.|INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité - (CRESS), METHODS team, Paris, France.|Paris Descartes University, Paris, France.|Centre d'Épidémiologie Clinique, Hôpital Hôtel-Dieu Escalier A2, 1 er étage, Assistance Publique des Hôpitaux de Paris, 1 Place du Parvis Notre-Dame, Paris, 75004, France.
Philippe
P
Ravaud
INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité - (CRESS), METHODS team, Paris, France. isabelle.boutron@htd.aphp.fr.|Paris Descartes University, Paris, France. isabelle.boutron@htd.aphp.fr.|Centre d'Épidémiologie Clinique, Hôpital Hôtel-Dieu Escalier A2, 1 er étage, Assistance Publique des Hôpitaux de Paris, 1 Place du Parvis Notre-Dame, Paris, 75004, France. isabelle.boutron@htd.aphp.fr.
Isabelle
I
Boutron
D035843:Biomedical Research / Q000592:standards*; D057189:Checklist; D000068598:Data Accuracy; D016208:Databases, Factual / Q000592:standards; D006801:Humans; D017712:Peer Review, Research / Q000592:standards*; D011642:Publications / Q000592:standards*; D016032:Randomized Controlled Trials as Topic; D058028:Research Report
Meta-analysis; Peer review process; Peer reviewers; Systematic review