Evaluation of PubMed filters used for evidence-based searching: validation using relative recall.
OBJECTIVE
The research sought to determine the value of PubMed filters and combinations of filters in literature selected for systematic reviews on therapy-related clinical questions.


METHODS
References to 35,281 included and 48,514 excluded articles were extracted from 2,629 reviews published prior to January 2008 in the Cochrane Database of Systematic Reviews and sent to PubMed with and without filters. Sensitivity, specificity, and precision were calculated from the percentages of unfiltered and filtered references retrieved for each review and averaged over all reviews.


RESULTS
Sensitivity of the Sensitive Clinical Queries filter was reasonable (92.7%, 92.1-93.3); specificity (16.1%, 15.1-17.1) and precision were low (49.5%, 48.5-50.5). The Specific Clinical Queries and the Single Term Medline Specific filters performed comparably (sensitivity, 78.2%, 77.2-79.2 vs. 78.0%; 77.0-79.0; specificity, 52.0%, 50.8-53.2 vs. 52.3%, 51.1-53.5; precision, 60.4%, 59.4-61.4 vs. 60.6%, 59.6-61.6). Combining the Abridged Index Medicus (AIM) and Single Term Medline Specific (65.2%, 63.8-66.6), Two Terms Medline Optimized (64.2%, 62.8-65.6), or Specific Clinical Queries filters (65.0%, 63.6-66.4) yielded the highest precision.


CONCLUSIONS
Sensitive and Specific Clinical Queries filters used to answer questions about therapy will result in a list of clinical trials but cannot be expected to identify only methodologically sound trials. The Specific Clinical Queries filters are not suitable for questions regarding therapy that cannot be answered with randomized controlled trials. Combining AIM with specific PubMed filters yields the highest precision in the Cochrane dataset.
Hoogendam
Arjen
A
Internal Medicine, Department of General Internal Medicine and Department of Medical Informatics, Radboud University Nijmegen Medical Centre, P.O. Box 9101, Nijmegen, The Netherlands. A.Hoogendam@aig.umcn.nl.
de Vries Robbé
Pieter F
PF
Stalenhoef
Anton F H
AF
Overbeke
A John P M
AJ
D000043:Abstracting and Indexing / Q000379:methods*; D019317:Evidence-Based Medicine; D016247:Information Storage and Retrieval / Q000379:methods* / Q000592:standards; D039781:PubMed