title: The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey.
abstract: BACKGROUND
  Randomized controlled trials (RCTs) are the most trustworthy source for evaluating treatment effects, but RCTs of spine surgery interventions often produce discordant results. The Fragility Index is a novel metric to inform about the robustness of statistically significant results.
  
  
  OBJECTIVE
  The aim was to determine the robustness of statistically significant results from RCTs of spine surgery interventions.
  
  
  METHODS
  This was a systematic survey.
  
  
  METHODS
  The sample included RCTs of spine surgery interventions.
  
  
  METHODS
  The Fragility Index is the minimum number of patients in a trial whose status would have to change from a nonevent to an event to change a statistically significant result to a nonsignificant result. Events refer to the occurrence of any dichotomous outcome, such as successful fusion, incident fracture, adjacent segment degeneration, or achievement of a certain functional score. A small Fragility Index indicates that the statistical significance of a result hinges on only a few events, and a large Fragility Index increases one's confidence in the observed treatment effects.
  
  
  METHODS
  We systematically reviewed a database for evidence-based orthopedics and identified all the RCTs that reported at least one positive outcome (ie, p<.05). Two reviewers independently assessed eligibility and extracted data. We used the Fisher exact test to compute Fragility Index values and multivariable linear regression to evaluate potential associated factors.
  
  
  RESULTS
  We identified 40 eligible RCTs with a median sample size of 132 patients (interquartile range [IQR] 79-208) and a median total number of outcome events for the chosen outcome of 31 (IQR 13-63). The median Fragility Index was two (IQR 1-3), which means that adding two events to one of the trial's treatment arms eliminated its statistical significance. The Fragility Index was less than or equal to three events in 75% of the trials, and was less than or equal to the number of patients lost to follow-up in 65% of the trials. Fragility Index values correlated positively with total sample size (r=0.35; p<.05). When adjusted for losses to follow-up and risk of bias, increasing Fragility Index values were associated only with increasingly significant reported p values (p<.01).
  
  
  CONCLUSIONS
  Statistically significant results in spine surgery RCTs are frequently fragile. The addition of only a small number of outcome events can completely eliminate significance. Surgeons, researchers, and other evidence users should exercise caution when interpreting the findings from RCTs with low Fragility Index values and applying these results to patient care.
authors.affiliation: Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada. Electronic address: nathan.evaniew@medportal.ca.
authors.forename: Nathan
authors.initials: N
authors.lastname: Evaniew
authors.affiliation: Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Carly
authors.initials: C
authors.lastname: Files
authors.affiliation: Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Christopher
authors.initials: C
authors.lastname: Smith
authors.affiliation: Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Mohit
authors.initials: M
authors.lastname: Bhandari
authors.affiliation: Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Michelle
authors.initials: M
authors.lastname: Ghert
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Michael
authors.initials: M
authors.lastname: Walsh
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Philip J
authors.initials: PJ
authors.lastname: Devereaux
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
authors.forename: Gordon
authors.initials: G
authors.lastname: Guyatt
mesh_terms: D006801:Humans; D019635:Neurosurgical Procedures / Q000009:adverse effects* / Q000706:statistics & numerical data; D019637:Orthopedic Procedures / Q000009:adverse effects* / Q000706:statistics & numerical data; D016032:Randomized Controlled Trials as Topic / Q000592:standards / Q000706:statistics & numerical data*; D013124:Spinal Injuries / Q000601:surgery*
keywords: Clinical epidemiology; Fragility Index; Outcomes; Randomized controlled trials; Spine surgery; Statistical significance