title: Optimism bias leads to inconclusive results-an empirical study.
abstract: OBJECTIVE
  Optimism bias refers to unwarranted belief in the efficacy of new therapies. We assessed the impact of optimism bias on a proportion of trials that did not answer their research question successfully and explored whether poor accrual or optimism bias is responsible for inconclusive results.
  
  
  METHODS
  Systematic review.
  
  
  METHODS
  Retrospective analysis of a consecutive-series phase III randomized controlled trials (RCTs) performed under the aegis of National Cancer Institute Cooperative groups.
  
  
  RESULTS
  Three hundred fifty-nine trials (374 comparisons) enrolling 150,232 patients were analyzed. Seventy percent (262 of 374) of the trials generated conclusive results according to the statistical criteria. Investigators made definitive statements related to the treatment preference in 73% (273 of 374) of studies. Investigators' judgments and statistical inferences were concordant in 75% (279 of 374) of trials. Investigators consistently overestimated their expected treatment effects but to a significantly larger extent for inconclusive trials. The median ratio of expected and observed hazard ratio or odds ratio was 1.34 (range: 0.19-15.40) in conclusive trials compared with 1.86 (range: 1.09-12.00) in inconclusive studies (P<0.0001). Only 17% of the trials had treatment effects that matched original researchers' expectations.
  
  
  CONCLUSIONS
  Formal statistical inference is sufficient to answer the research question in 75% of RCTs. The answers to the other 25% depend mostly on subjective judgments, which at times are in conflict with statistical inference. Optimism bias significantly contributes to inconclusive results.
authors.affiliation: Department of Medicine, Center for Evidence-based Medicine & Health Outcomes Research, University of South Florida, Tampa, FL 33612, USA. bdjulbeg@health.usf.edu
authors.forename: Benjamin
authors.initials: B
authors.lastname: Djulbegovic
authors.forename: Ambuj
authors.initials: A
authors.lastname: Kumar
authors.forename: Anja
authors.initials: A
authors.lastname: Magazin
authors.forename: Anneke T
authors.initials: AT
authors.lastname: Schroen
authors.forename: Heloisa
authors.initials: H
authors.lastname: Soares
authors.forename: Iztok
authors.initials: I
authors.lastname: Hozo
authors.forename: Mike
authors.initials: M
authors.lastname: Clarke
authors.forename: Daniel
authors.initials: D
authors.lastname: Sargent
authors.forename: Michael J
authors.initials: MJ
authors.lastname: Schell
mesh_terms: D017326:Clinical Trials, Phase III as Topic / Q000592:standards*; D036262:Empirical Research; D015340:Epidemiologic Research Design; D006801:Humans; D017594:Publication Bias; D016032:Randomized Controlled Trials as Topic / Q000592:standards*