The evidence provided by a single trial is less reliable than its statistical analysis suggests.
OBJECTIVE
To investigate whether a single trial can provide sufficiently robust evidence to warrant clinical implementation of its results. Trial-specific factors, such as subject selection, study design, and execution strategy, have an impact on the outcome of trials. In multiple trials, they may lead to heterogeneity that can be taken into account in the (random effects) meta-analysis. Single trials lack this method of estimating the impact of such factors, and this affects the credibility of the results.


METHODS
To indicate how much the precision of the results of a single trial might be overestimated, we calculated the ratio of the widths of the confidence intervals when heterogeneity was taken into account and when it was not.


RESULTS
The ratios of the widths of the confidence intervals with and without between-study variability were 1.15, 1.41, and 2.00, when the heterogeneity I(2) values were 0.25, 0.50, and 0.75, respectively.


CONCLUSIONS
The results of a single trial should be interpreted with caution. When it is difficult to predict or determine how trial-specific factors influence the results, the best way to evaluate the performance of a treatment is to use multiple, possibly smaller, trials.
Borm
George F
GF
Department of Epidemiology, Biostatistics and HTA, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands. g.borm@ebh.umcn.nl
Lemmers
Oscar
O
Fransen
Jaap
J
Donders
Rogier
R
D018501:Antirheumatic Agents / Q000627:therapeutic use; D001172:Arthritis, Rheumatoid / Q000188:drug therapy; D002986:Clinical Trials as Topic / Q000592:standards*; D003627:Data Interpretation, Statistical; D004574:Electromagnetic Fields / Q000009:adverse effects; D019317:Evidence-Based Medicine / Q000592:standards*; D006801:Humans; D007953:Leukemia, Radiation-Induced / Q000453:epidemiology / Q000209:etiology; D017063:Outcome Assessment, Health Care / Q000379:methods; D018579:Patient Selection; D012107:Research Design
D018501:Antirheumatic Agents