title: Analyzing 10 years of early awareness and alert activity in the United kingdom.
abstract: OBJECTIVE
  The aim of this study was to assess the accuracy of the English National Horizon Scanning Centre (NHSC) in identifying and filtering pharmaceutical developments using end user and international collaborator databases of emerging technologies as proxies for new drugs of likely significance to health services and/or patients.
  
  
  METHODS
  We used the NHSC information system and the list of National Institute for Health and Clinical Excellence (NICE) technology appraisals to estimate the false positive rate for NHSC identification, filtration, and reporting. We assessed the sensitivity of NHSC identification and filtration of pharmaceuticals for NICE technology appraisals from 1999 to the end of December 2010, and for pharmaceuticals entered into the EuroScan International Network database.
  
  
  RESULTS
  We estimate that overall NHSC identification, filtration and reporting had a positive predictive value of 0.39 (95 percent CI, 0.36 to 0.43) and a false positive rate of 60 percent. Using NICE appraisals and EuroScan's database as proxies for pharmaceuticals of significance, we estimate the NHSC sensitivity over the 10-year period at 0.92 (95 percent CI, 0.89 to 0.95) and 0.89 (95 percent CI, 0.82 to 0.96) respectively.
  
  
  CONCLUSIONS
  Our results suggest that the NHSC has performed well in terms of sensitivity over the past decade, but that the false positive rate of 60 percent may indicate that the filtration criteria for pharmaceuticals could be tightened for increased efficiency. Future evaluations of EAA systems should include an element of external review and explore the level of accuracy acceptable to funders and customers of such systems.
authors.affiliation: University of Birmingham, United Kingdom. c.packer@bham.ac.uk
authors.forename: Claire
authors.initials: C
authors.lastname: Packer
authors.forename: Matthew
authors.initials: M
authors.lastname: Fung
authors.forename: Andrew
authors.initials: A
authors.lastname: Stevens
mesh_terms: D001364:Awareness; D020811:Biomedical Technology / Q000639:trends; D003625:Data Collection / Q000379:methods; D016208:Databases, Factual; D033181:Information Dissemination; D004364:Pharmaceutical Preparations; D015397:Program Evaluation; D012680:Sensitivity and Specificity; D013673:Technology Assessment, Biomedical; D006113:United Kingdom
chemical_list: D004364:Pharmaceutical Preparations