Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions.
OBJECTIVE
Searching multiple sources when conducting systematic reviews is considered good practice. We aimed to investigate the impact of using sources beyond PubMed in systematic reviews of therapeutic interventions.


METHODS
We randomly selected 50 Cochrane reviews that searched the PubMed (or MEDLINE) and EMBASE databases and included a meta-analysis of ≥10 studies. We checked whether each eligible record in each review (n = 2,700) was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of ≥10 studies in each review, we examined whether excluding studies not found in PubMed affected results.


RESULTS
A median of one record per review was indexed in EMBASE but not in PubMed; a median of four records per review was not indexed in PubMed or EMBASE. Meta-analyses included a median of 13.5 studies; a median of zero studies per meta-analysis was indexed in EMBASE but not in PubMed; a median of one study per meta-analysis was not indexed in PubMed or EMBASE. Meta-analysis using only PubMed-indexed vs. all available studies led to a different conclusion in a single case (on the basis of conventional criteria for statistical significance). In meta-regression analyses, effects in PubMed- vs. non-PubMed-indexed studies were statistically significantly different in a single data set.


CONCLUSIONS
For systematic reviews of the effects of therapeutic interventions, gains from searching sources beyond PubMed, and from searching EMBASE in particular are modest.
Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Biostatistics, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
Christopher W
CW
Halladay
Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Health Services, Policy, and Practice, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
Thomas A
TA
Trikalinos
Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
Ian T
IT
Schmid
Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Biostatistics, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
Christopher H
CH
Schmid
Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Health Services, Policy, and Practice, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA. Electronic address: issa_dahabreh@brown.edu.
Issa J
IJ
Dahabreh
D015706:Bibliometrics; D016206:Databases, Bibliographic; D006801:Humans; D015201:Meta-Analysis as Topic; D039781:PubMed; D012107:Research Design; D012196:Review Literature as Topic
Databases; EMBASE; Health technology assessment; Literature search; Meta-analysis; PubMed; Rapid review; Systematic review