Meta-analyses with binary outcomes: how many studies need to be omitted to detect a publication bias?
Meta- and pooled analyses are increasingly applied to aggregate the results of a number of studies, especially in health sciences. A typical difficulty is the presence of a publication bias. Usually Egger's regression test and funnel plots are applied to detect such a publication bias. A simulation study was conducted to investigate the quantity of null and negative results required to be omitted to detect a publication bias. In particular, the performance of Egger's test and funnel plots was considered in two scenarios with binary outcomes and expected odds ratios (OR) of 1 and 2, respectively. For both scenarios Egger's test detected only a small fraction of publication biases if few studies were deleted, corresponding to the results of a random deletion. Moreover, if a true null effect is present Egger's test is quite unlikely to detect a publication bias even if a considerable proportion of the null results are missing. Generally, the detection of a publication bias using Egger's test is only likely if both the "true" effect and the bias are large enough. Visual inspection of the funnel plots resulted in a higher fraction of detected publication biases in cases where a bias was present and in cases where studies were randomly deleted, revealing the arbitrariness of this method. Evidence indicates that standard methods for detection of a publication bias do not necessarily detect such a bias; thus, additional tests for publication bias need to be applied.
Fakultat Statistik, Technische Universitat Dortmund, Dortmund, Germany.
Evgenia
E
Saveleva
Silvia
S
Selinski
D003198:Computer Simulation; D016014:Linear Models; D015201:Meta-Analysis as Topic; D016017:Odds Ratio; D017594:Publication Bias / Q000706:statistics & numerical data*; D012044:Regression Analysis; D012106:Research / Q000706:statistics & numerical data