Publication bias was not a good reason to discourage trials with low power.
OBJECTIVE
The objective was to investigate whether it is justified to discourage trials with less than 80% power. Trials with low power are unlikely to produce conclusive results, but their findings can be used by pooling then in a meta-analysis. However, such an analysis may be biased, because trials with low power are likely to have a nonsignificant result and are less likely to be published than trials with a statistically significant outcome.


METHODS
We simulated several series of studies with varying degrees of publication bias and then calculated the "real" one-sided type I error and the bias of meta-analyses with a "nominal" error rate (significance level) of 2.5%.


RESULTS
In single trials, in which heterogeneity was set at zero, low, and high, the error rates were 2.3%, 4.7%, and 16.5%, respectively. In multiple trials with 80%-90% power and a publication rate of 90% when the results were nonsignificant, the error rates could be as high as 5.1%. When the power was 50% and the publication rate of non-significant results was 60%, the error rates did not exceed 5.3%, whereas the bias was at most 15% of the difference used in the power calculation.


CONCLUSIONS
The impact of publication bias does not warrant the exclusion of trials with 50% power.
Borm
George F
GF
Department of Epidemiology and Biostatistics, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands. g.borm@epib.umcn.nl
den Heijer
Martin
M
Zielhuis
Gerhard A
GA
D015982:Bias; D002986:Clinical Trials as Topic / Q000706:statistics & numerical data*; D016021:Epidemiologic Studies; D004989:Ethics; D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias / Q000706:statistics & numerical data*; D015203:Reproducibility of Results