title: Development and Validation of a Standardized Tool for Prioritization of Information Sources.
abstract: OBJECTIVE
  To validate the utility and effectiveness of a standardized tool for prioritization of information sources for early detection of diseases.
  
  
  METHODS
  The tool was developed with input from diverse public health experts garnered through survey. Ten raters used the tool to evaluate ten information sources and reliability among raters was computed. The Proc mixed procedure with random effect statement and SAS Macros were used to compute multiple raters' Fleiss Kappa agreement and Kendall's Coefficient of Concordance.
  
  
  RESULTS
  Ten disparate information sources evaluated obtained the following composite scores: ProMed 91%; WAHID 90%; Eurosurv 87%; MediSys 85%; SciDaily 84%; EurekAl 83%; CSHB 78%; GermTrax 75%; Google 74%; and CBC 70%. A Fleiss Kappa agreement of 50.7% was obtained for ten information sources and 72.5% for a sub-set of five sources rated, which is substantial agreement validating the utility and effectiveness of the tool.
  
  
  CONCLUSIONS
  This study validated the utility and effectiveness of a standardized criteria tool developed to prioritize information sources. The new tool was used to identify five information sources suited for use by the KIWI system in the CEZD-IIR project to improve surveillance of infectious diseases. The tool can be generalized to situations when prioritization of numerous information sources is necessary.
authors.affiliation: Canadian Network for Public Health Intelligence, National Microbiology Laboratory, Winnipeg, Manitoba.
authors.forename: Holy
authors.initials: H
authors.lastname: Akwar
authors.affiliation: Canadian Food Inspection Agency, Owen Sound, Ontario.
authors.forename: Harold
authors.initials: H
authors.lastname: Kloeze
authors.affiliation: Canadian Network for Public Health Intelligence, National Microbiology Laboratory, Winnipeg, Manitoba.
authors.forename: Shamir
authors.initials: S
authors.lastname: Mukhi
keywords: Early Warning; Emerging Pathogens; Intelligence; Response; Zoonotic Diseases