Obtaining evidence by a single well-powered trial or several modestly powered trials.
There is debate whether clinical trials with suboptimal power are justified and whether results from large studies are more reliable than the (combined) results of smaller trials. We quantified the error rates for evaluations based on single conventionally powered trials (80% or 90% power) versus evaluations based on the random-effects meta-analysis of a series of smaller trials. When a treatment was assumed to have no effect but heterogeneity was present, the error rates for a single trial were increased more than 10-fold above the nominal rate, even for low heterogeneity. Conversely, for meta-analyses on a series of trials, the error rates were correct. When selective publication was present, the error rates were always increased, but they still tended to be lower for a series of trials than single trials. We conclude that evidence of efficacy based on a series of (smaller) trials, may lower the error rates compared with using a single well-powered trial. Only when both heterogeneity and selective publication can be excluded, a single trial is able to provide conclusive evidence.
Department of Epidemiology, Biostatistics and HTA, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands j.IntHout@ebh.umcn.nl.
Joanna
J
IntHout
Stanford Prevention Research Center, Department of Medicine and Department of Health Research and Policy, Stanford University School of Medicine, CA, USA Department of Statistics, Stanford University School of Humanities and Sciences, CA, USA.
John Pa
JP
Ioannidis
Department of Epidemiology, Biostatistics and HTA, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands.
George F
GF
Borm
D002986:Clinical Trials as Topic / Q000379:methods*; D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias; D012107:Research Design