title: Evaluating the utility of syndromic surveillance algorithms for screening to detect potentially clonal hospital infection outbreaks.
abstract: OBJECTIVE
  The authors evaluated algorithms commonly used in syndromic surveillance for use as screening tools to detect potentially clonal outbreaks for review by infection control practitioners.
  
  
  METHODS
  Study phase 1 applied four aberrancy detection algorithms (CUSUM, EWMA, space-time scan statistic, and WSARE) to retrospective microbiologic culture data, producing a list of past candidate outbreak clusters. In phase 2, four infectious disease physicians categorized the phase 1 algorithm-identified clusters to ascertain algorithm performance. In phase 3, project members combined the algorithms to create a unified screening system and conducted a retrospective pilot evaluation.
  
  
  METHODS
  The study calculated recall and precision for each algorithm, and created precision-recall curves for various methods of combining the algorithms into a unified screening tool.
  
  
  RESULTS
  Individual algorithm recall and precision ranged from 0.21 to 0.31 and from 0.053 to 0.29, respectively. Few candidate outbreak clusters were identified by more than one algorithm. The best method of combining the algorithms yielded an area under the precision-recall curve of 0.553. The phase 3 combined system detected all infection control-confirmed outbreaks during the retrospective evaluation period.
  
  
  CONCLUSIONS
  Lack of phase 2 reviewers' agreement indicates that subjective expert review was an imperfect gold standard. Less conservative filtering of culture results and alternate parameter selection for each algorithm might have improved algorithm performance.
  
  
  CONCLUSIONS
  Hospital outbreak detection presents different challenges than traditional syndromic surveillance. Nevertheless, algorithms developed for syndromic surveillance have potential to form the basis of a combined system that might perform clinically useful hospital outbreak screening.
authors: Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA. randy.carnevale@vanderbilt.edu
authors: Randy J
authors: RJ
authors: Carnevale
authors: Thomas R
authors: TR
authors: Talbot
authors: William
authors: W
authors: Schaffner
authors: Karen C
authors: KC
authors: Bloch
authors: Titus L
authors: TL
authors: Daniels
authors: Randolph A
authors: RA
authors: Miller
references: J Am Med Inform Assoc. 2009 Nov-Dec;16(6):855-63
references: 19717809
references: AMIA Annu Symp Proc. 2008 Nov 06;:611-5
references: 18999304
references: Am J Epidemiol. 1997 Jul 15;146(2):161-70
references: 9230778
references: J Clin Microbiol. 1994 Feb;32(2):407-15
references: 7908673
references: MMWR Suppl. 2004 Sep 24;53:32-9
references: 15714624
references: J Biomed Inform. 2005 Apr;38(2):99-113
references: 15797000
references: J Clin Microbiol. 2004 Mar;42(3):1170-5
references: 15004070
references: Emerg Infect Dis. 2008 Feb;14(2):317-9
references: 18258130
references: J Am Med Inform Assoc. 2003 Sep-Oct;10(5):399-408
references: 12807803
references: Emerg Infect Dis. 2001 Nov-Dec;7(6):933-44
references: 11747719
references: AMIA Annu Symp Proc. 2007 Oct 11;:369-73
references: 18693860
references: Methods Inf Med. 2004;43(5):483-5
references: 15702206
references: J Am Med Inform Assoc. 2003 Nov-Dec;10(6):547-54
references: 12925547
references: PLoS Med. 2005 Mar;2(3):e59
references: 15719066
references: Emerg Infect Dis. 2003 Jun;9(6):641-6
references: 12781002
references: J Biomed Inform. 2007 Aug;40(4):370-9
references: 17095301
references: Ann Intern Med. 2004 Jun 1;140(11):910-22
references: 15172906