title: Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial.
abstract: BACKGROUND
  If positive-outcome bias exists, it threatens the integrity of evidence-based medicine.
  
  
  METHODS
  We sought to determine whether positive-outcome bias is present during peer review by testing whether peer reviewers would (1) recommend publication of a "positive" version of a fabricated manuscript over an otherwise identical "no-difference" version, (2) identify more purposefully placed errors in the no-difference version, and (3) rate the "Methods" section in the positive version more highly than the identical "Methods" section in the no-difference version. Two versions of a well-designed randomized controlled trial that differed only in the direction of the finding of the principal study end point were submitted for peer review to 2 journals in 2008-2009. Of 238 reviewers for The Journal of Bone and Joint Surgery and Clinical Orthopaedics and Related Research randomly allocated to review either a positive or a no-difference version of the manuscript, 210 returned reviews.
  
  
  RESULTS
  Reviewers were more likely to recommend the positive version of the test manuscript for publication than the no-difference version (97.3% vs 80.0%, P < .001). Reviewers detected more errors in the no-difference version than in the positive version (0.85 vs 0.41, P < .001). Reviewers awarded higher methods scores to the positive manuscript than to the no-difference manuscript (8.24 vs 7.53, P = .005), although the "Methods" sections in the 2 versions were identical.
  
  
  CONCLUSIONS
  Positive-outcome bias was present during peer review. A fabricated manuscript with a positive outcome was more likely to be recommended for publication than was an otherwise identical no-difference manuscript.
authors.affiliation: Department of Orthopaedics, University of Washington, Seattle, 98195-6500, USA.
authors.forename: Gwendolyn B
authors.initials: GB
authors.lastname: Emerson
authors.forename: Winston J
authors.initials: WJ
authors.lastname: Warme
authors.forename: Fredric M
authors.initials: FM
authors.lastname: Wolf
authors.forename: James D
authors.initials: JD
authors.lastname: Heckman
authors.forename: Richard A
authors.initials: RA
authors.lastname: Brand
authors.forename: Seth S
authors.initials: SS
authors.lastname: Leopold
mesh_terms: D006801:Humans; D016015:Logistic Models; D017063:Outcome Assessment, Health Care; D017712:Peer Review, Research; D010506:Periodicals as Topic; D017594:Publication Bias; D016032:Randomized Controlled Trials as Topic