The risk of false-positive results in orthopaedic surgical trials.
The risk of concluding that the results of a particular study are true, when, in fact, they really are attributable to chance (or random sampling error) is underappreciated by investigators. This erroneous false-positive conclusion is designated as a Type I or alpha error. The extent to which randomized trials in surgery risk Type I errors is unclear. The current authors hand-searched four orthopaedic journals, six general surgery journals, and five medical journals to identify recently published randomized trials (within the past 2 years). Information on outcomes and statistical adjustment for multiple outcomes was recorded for each study. The risk of a Type I error was calculated for each study that did not explicitly state a primary outcome measure for the main statistical comparison. One hundred fifty-nine studies met the inclusion criteria for the study: 60 studies from orthopaedic journals, 49 studies from nonorthopaedic surgical journals, and 50 studies from medical journals. Of the trials that did not state a primary outcome measure, the risk of Type I errors (false-positive results) in orthopaedic and nonorthopaedic surgery journals (mean 37.3% +/- 13.3% and 37.6% +/- 10.5%, respectively) were significantly greater than medical journals (10.1% +/- 1.9%). In the current review of randomized trials in surgery and medicine, the following is reported: (1) reporting of primary outcomes in trials was inadequate; (2) one in three trials in surgery and one in 10 trials in medicine risked false-positive results; and (3) few trials in surgery and medicine considered adjustment for multiple comparisons.
Bhandari
Mohit
M
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1200 Main Street West, Room 2C3, Hamilton, Ontario L8N 3Z5, Canada. bhandari@sympatico.ca
Whang
William
W
Kuo
Jonathann C
JC
Devereaux
P J
PJ
Sprague
Sheila
S
Tornetta
Paul
P
D006801:Humans; D009985:Orthopedics; D010506:Periodicals as Topic / Q000706:statistics & numerical data*; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D015203:Reproducibility of Results