title: The risk of false-positive results in orthopaedic surgical trials.
abstract: The risk of concluding that the results of a particular study are true, when, in fact, they really are attributable to chance (or random sampling error) is underappreciated by investigators. This erroneous false-positive conclusion is designated as a Type I or alpha error. The extent to which randomized trials in surgery risk Type I errors is unclear. The current authors hand-searched four orthopaedic journals, six general surgery journals, and five medical journals to identify recently published randomized trials (within the past 2 years). Information on outcomes and statistical adjustment for multiple outcomes was recorded for each study. The risk of a Type I error was calculated for each study that did not explicitly state a primary outcome measure for the main statistical comparison. One hundred fifty-nine studies met the inclusion criteria for the study: 60 studies from orthopaedic journals, 49 studies from nonorthopaedic surgical journals, and 50 studies from medical journals. Of the trials that did not state a primary outcome measure, the risk of Type I errors (false-positive results) in orthopaedic and nonorthopaedic surgery journals (mean 37.3% +/- 13.3% and 37.6% +/- 10.5%, respectively) were significantly greater than medical journals (10.1% +/- 1.9%). In the current review of randomized trials in surgery and medicine, the following is reported: (1) reporting of primary outcomes in trials was inadequate; (2) one in three trials in surgery and one in 10 trials in medicine risked false-positive results; and (3) few trials in surgery and medicine considered adjustment for multiple comparisons.
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, 1200 Main Street West, Room 2C3, Hamilton, Ontario L8N 3Z5, Canada. bhandari@sympatico.ca
authors.forename: Mohit
authors.initials: M
authors.lastname: Bhandari
authors.forename: William
authors.initials: W
authors.lastname: Whang
authors.forename: Jonathann C
authors.initials: JC
authors.lastname: Kuo
authors.forename: P J
authors.initials: PJ
authors.lastname: Devereaux
authors.forename: Sheila
authors.initials: S
authors.lastname: Sprague
authors.forename: Paul
authors.initials: P
authors.lastname: Tornetta
mesh_terms: D006801:Humans; D009985:Orthopedics; D010506:Periodicals as Topic / Q000706:statistics & numerical data*; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D015203:Reproducibility of Results