Meta-analyses of small numbers of trials often agree with longer-term results.
OBJECTIVE
Many systematic reviews include only a few studies. It is unclear whether recommendations based on these will be correct in the longer term; hence, this article explores whether meta-analyses give reliable results after only a few studies.


METHODS
Cumulative meta-analysis of data from 65 meta-analyses from 18 Cochrane systematic reviews was carried out. Various measures of closeness to the pooled estimate from all trials after three and five trials were included. Changes during the accumulation of evidence were noted.


RESULTS
The 95% confidence interval included the final estimate in 72% of meta-analyses after three studies and in 83% after five studies. It took a median of four (interquartile range: 1.25-6) studies to get within 10% of the final point estimate. Agreement between the results at three and five studies and the final estimate was not predicted by the number of participants, the number of events, τ(2), or I(2). Estimates could still change substantially after many trials were included.


CONCLUSIONS
Many of the conclusions drawn from systematic reviews with small numbers of included studies will be correct in the long run, but it is not possible to predict which ones.
Herbison
Peter
P
Department of Preventive and Social Medicine, University of Otago, P.O. Box 913, Dunedin 9054, New Zealand. peter.herbison@otago.ac.nz
Hay-Smith
Jean
J
Gillespie
William J
WJ
D002986:Clinical Trials as Topic / Q000592:standards*; D003627:Data Interpretation, Statistical; D006801:Humans; D015201:Meta-Analysis as Topic; D011785:Quality Assurance, Health Care / Q000592:standards*; D015203:Reproducibility of Results; D012196:Review Literature as Topic