Publication bias and meta-analyses: a practical example.
OBJECTIVE
Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.


METHODS
We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.


RESULTS
The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.


CONCLUSIONS
We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them.
Burdett
Sarah
S
Meta-analysis Group, Medical Research Council Clinical Trials Unit, London, UK. sb@ctu.mrc.ac.uk
Stewart
Lesley A
LA
Tierney
Jayne F
JF
D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias; D016032:Randomized Controlled Trials as Topic; D006113:United Kingdom