title: Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE.
abstract: BACKGROUND
  A systematic and extensive search for as many eligible studies as possible is essential in any systematic review. When searching for diagnostic test accuracy (DTA) studies in bibliographic databases, it is recommended that terms for disease (target condition) are combined with terms for the diagnostic test (index test). Researchers have developed methodological filters to try to increase the precision of these searches. These consist of text words and database indexing terms and would be added to the target condition and index test searches.Efficiently identifying reports of DTA studies presents challenges because the methods are often not well reported in their titles and abstracts, suitable indexing terms may not be available and relevant indexing terms do not seem to be consistently assigned. A consequence of using search filters to identify records for diagnostic reviews is that relevant studies might be missed, while the number of irrelevant studies that need to be assessed may not be reduced. The current guidance for Cochrane DTA reviews recommends against the addition of a methodological search filter to target condition and index test search, as the only search approach.
  
  
  OBJECTIVE
  To systematically review empirical studies that report the development or evaluation, or both, of methodological search filters designed to retrieve DTA studies in MEDLINE and EMBASE.
  
  
  METHODS
  We searched MEDLINE (1950 to week 1 November 2012); EMBASE (1980 to 2012 Week 48); the Cochrane Methodology Register (Issue 3, 2012); ISI Web of Science (11 January 2013); PsycINFO (13 March 2013); Library and Information Science Abstracts (LISA) (31 May 2010); and Library, Information Science & Technology Abstracts (LISTA) (13 March 2013). We undertook citation searches on Web of Science, checked the reference lists of relevant studies, and searched the Search Filters Resource website of the InterTASC Information Specialists' Sub-Group (ISSG).
  
  
  METHODS
  Studies reporting the development or evaluation, or both, of a MEDLINE or EMBASE search filter aimed at retrieving DTA studies, which reported a measure of the filter's performance were eligible.
  
  
  METHODS
  The main outcome was a measure of filter performance, such as sensitivity or precision. We extracted data on the identification of the reference set (including the gold standard and, if used, the non-gold standard records), how the reference set was used and any limitations, the identification and combination of the search terms in the filters, internal and external validity testing, the number of filters evaluated, the date the study was conducted, the date the searches were completed, and the databases and search interfaces used. Where 2 x 2 data were available on filter performance, we used these to calculate sensitivity, specificity, precision and Number Needed to Read (NNR), and 95% confidence intervals (CIs). We compared the performance of a filter as reported by the original development study and any subsequent studies that evaluated the same filter.
  
  
  RESULTS
  Ninteen studies were included, reporting on 57 MEDLINE filters and 13 EMBASE filters. Thirty MEDLINE and four EMBASE filters were tested in an evaluation study where the performance of one or more filters was tested against one or more gold standards. The reported outcome measures varied. Some studies reported specificity as well as sensitivity if a reference set containing non-gold standard records in addition to gold standard records was used. In some cases, the original development study did not report any performance data on the filters. Original performance from the development study was not available for 17 filters that were subsequently tested in evaluation studies. All 19 studies reported the sensitivity of the filters that they developed or evaluated, nine studies reported the specificities and 14 studies reported the precision.No filter which had original performance data from its development study, and was subsequently tested in an evaluation study, had what we defined a priori as acceptable sensitivity (> 90%) and precision (> 10%). In studies that developed MEDLINE filters that were evaluated in another study (n = 13), the sensitivity ranged from 55% to 100% (median 86%) and specificity from 73% to 98% (median 95%). Estimates of performance were lower in eight studies that evaluated the same 13 MEDLINE filters, with sensitivities ranging from 14% to 100% (median 73%) and specificities ranging from 15% to 96% (median 81%). Precision ranged from 1.1% to 40% (median 9.5%) in studies that developed MEDLINE filters and from 0.2% to 16.7% (median 4%) in studies that evaluated these filters. A similar range of specificities and precision were reported amongst the evaluation studies for MEDLINE filters without an original performance measure. Sensitivities ranged from 31% to 100% (median 71%), specificity ranged from 13% to 90% (median 55.5%) and precision from 1.0% to 11.0% (median 3.35%).For the EMBASE filters, the original sensitivities reported in two development studies ranged from 74% to 100% (median 90%) for three filters, and precision ranged from 1.2% to 17.6% (median 3.7%). Evaluation studies of these filters had sensitivities from 72% to 97% (median 86%) and precision from 1.2% to 9% (median 3.7%). The performance of EMBASE search filters in development and evaluation studies were more alike than the performance of MEDLINE filters in development and evaluation studies. None of the EMBASE filters in either type of study had a sensitivity above 90% and precision above 10%.
  
  
  CONCLUSIONS
  None of the current methodological filters designed to identify reports of primary DTA studies in MEDLINE or EMBASE combine sufficiently high sensitivity, required for systematic reviews, with a reasonable degree of precision. This finding supports the current recommendation in the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy that the combination of methodological filter search terms with terms for the index test and target condition should not be used as the only approach when conducting formal searches to inform systematic reviews of DTA.
authors.affiliation: School of Social and Community Medicine, Canynge Hall, 39 Whatley Road, Bristol, UK, BS8 2PS.
authors.forename: Rebecca
authors.initials: R
authors.lastname: Beynon
authors.forename: Mariska M G
authors.initials: MM
authors.lastname: Leeflang
authors.forename: Steve
authors.initials: S
authors.lastname: McDonald
authors.forename: Anne
authors.initials: A
authors.lastname: Eisinga
authors.forename: Ruth L
authors.initials: RL
authors.lastname: Mitchell
authors.forename: Penny
authors.initials: P
authors.lastname: Whiting
authors.forename: Julie M
authors.initials: JM
authors.lastname: Glanville
mesh_terms: D016206:Databases, Bibliographic; D003933:Diagnosis; D016247:Information Storage and Retrieval / Q000379:methods* / Q000592:standards; D016239:MEDLINE; D012015:Reference Standards; D012196:Review Literature as Topic; D057386:Search Engine; D012680:Sensitivity and Specificity; D013358:Subject Headings
references.citation: BMJ. 2005 May 21;330(7501):1179
references.pmid: 15894554
references.citation: Proc Annu Symp Comput Appl Med Care. 1995;:436-40
references.pmid: 8563319
references.citation: BMJ. 2004 May 1;328(7447):1040
references.pmid: 15073027
references.citation: Health Info Libr J. 2007 Sep;24(3):188-92
references.pmid: 17714173
references.citation: J Clin Epidemiol. 2000 Jan;53(1):65-9
references.pmid: 10693905
references.citation: Ann Clin Biochem. 2002 Jul;39(Pt 4):345-50
references.pmid: 12117438
references.citation: Cochrane Database Syst Rev. 2011 Aug 10;(8):MR000026
references.pmid: 21833989
references.citation: J Clin Epidemiol. 2009 Sep;62(9):974-81
references.pmid: 19230607
references.citation: J Clin Epidemiol. 2006 Jul;59(7):710-4
references.pmid: 16765274
references.citation: J Med Libr Assoc. 2008 Oct;96(4):356-61
references.pmid: 18974813
references.citation: BMC Med. 2004 Jun 09;2:23
references.pmid: 15189561
references.citation: Clin Chem. 2003 Jan;49(1):7-18
references.pmid: 12507954
references.citation: J Clin Epidemiol. 2006 Mar;59(3):234-40
references.pmid: 16488353
references.citation: J Clin Epidemiol. 2011 Jun;64(6):602-7
references.pmid: 21075596
references.citation: Fam Pract. 1997 Jun;14(3):204-8
references.pmid: 9201493
references.citation: BMC Med Res Methodol. 2002 Jul 03;2:9
references.pmid: 12097142
references.citation: J Am Med Inform Assoc. 2002 Nov-Dec;9(6):653-8
references.pmid: 12386115
references.citation: BMC Med. 2005 Mar 29;3:7
references.pmid: 15796772
references.citation: BMC Med Inform Decis Mak. 2005 Mar 22;5:8
references.pmid: 15784134
references.citation: J Clin Epidemiol. 2005 May;58(5):444-9
references.pmid: 15845330
references.citation: J Am Med Inform Assoc. 2005 Jul-Aug;12(4):481-5
references.pmid: 15802476
references.citation: Ned Tijdschr Geneeskd. 2002 Nov 30;146(48):2281-4
references.pmid: 12497754
references.citation: J Med Libr Assoc. 2003 Jul;91(3):341-6
references.pmid: 12883560
references.citation: AMIA Annu Symp Proc. 2003;:719-23
references.pmid: 14728267
references.citation: Proc Annu Symp Comput Appl Med Care. 1994;:905-9
references.pmid: 7950055
references.citation: Radiology. 2008 May;247(2):365-73
references.pmid: 18372447
references.citation: Int J Nurs Terminol Classif. 2005 Apr-Jun;16(2):26-32
references.pmid: 16045550
references.citation: Proc Annu Symp Comput Appl Med Care. 1993;:601-5
references.pmid: 8130545
references.citation: AMIA Annu Symp Proc. 2007 Oct 11;:801-5
references.pmid: 18693947
references.citation: Health Info Libr J. 2003 Sep;20(3):150-9
references.pmid: 12919278
references.citation: J Am Med Inform Assoc. 1994 Nov-Dec;1(6):447-58
references.pmid: 7850570