Variation in results from randomized, controlled trials: stochastic or systematic?
OBJECTIVE
Randomized controlled trials (RCTs) are considered the highest grade of research evidence, yet properly conducted trials investigating the same association often yield conflicting results. Our objective was to assess whether variability in treatment protocols of RCTs investigating the same topic could explain distinct patterns of outcomes.


METHODS
A review of meta-analyses identified clinical topics involving RCTs with variable pharmacologic dosing and disparate outcomes. Topics were retained if at least two pairs of trials had results suggesting contradictory yet strong exposure-outcome associations.


RESULTS
The search yielded 6 clinical topics and 58 RCTs, and individual RCTs were classified into two groups, based on low and high dosages of the intervention. Aggregate odds ratios for studies in the low- and high-dose groups were often substantially discordant. For example, odds ratios were 1.76 (95% confidence interval [CI]=1.02-3.03) for low-dose and 0.56 (95% CI=0.31-1.03) for high-dose trials evaluating low-molecular weight heparin and pulmonary embolism. In an exploratory analysis, outcomes for low- and high-dose groups in the comparison arms of trials (including patients assigned to placebo) had statistically significant differences in four of five analyzable topics, suggesting differences in patient characteristics across trials.


CONCLUSIONS
Conflicting results from RCTs can represent a spectrum of "real" outcomes for specific treatments. Such trials are best evaluated by considering concurrently both the validity of study design as well as the generalizability of patients and interventions involved.
Jane-wit
Daniel
D
Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, USA.
Horwitz
Ralph I
RI
Concato
John
J
D000368:Aged; D000369:Aged, 80 and over; D004334:Drug Administration Schedule; D004358:Drug Therapy / Q000379:methods; D019317:Evidence-Based Medicine / Q000379:methods / Q000592:standards; D005260:Female; D006801:Humans; D008297:Male; D015201:Meta-Analysis as Topic; D008875:Middle Aged; D016017:Odds Ratio; D016032:Randomized Controlled Trials as Topic / Q000379:methods / Q000592:standards*; D015203:Reproducibility of Results; D012107:Research Design; D016896:Treatment Outcome