title: Vaccine adverse event text mining system for extracting features from vaccine safety reports.
abstract: OBJECTIVE
  To develop and evaluate a text mining system for extracting key clinical features from vaccine adverse event reporting system (VAERS) narratives to aid in the automated review of adverse event reports.
  
  
  METHODS
  Based upon clinical significance to VAERS reviewing physicians, we defined the primary (diagnosis and cause of death) and secondary features (eg, symptoms) for extraction. We built a novel vaccine adverse event text mining (VaeTM) system based on a semantic text mining strategy. The performance of VaeTM was evaluated using a total of 300 VAERS reports in three sequential evaluations of 100 reports each. Moreover, we evaluated the VaeTM contribution to case classification; an information retrieval-based approach was used for the identification of anaphylaxis cases in a set of reports and was compared with two other methods: a dedicated text classifier and an online tool.
  
  
  METHODS
  The performance metrics of VaeTM were text mining metrics: recall, precision and F-measure. We also conducted a qualitative difference analysis and calculated sensitivity and specificity for classification of anaphylaxis cases based on the above three approaches.
  
  
  RESULTS
  VaeTM performed best in extracting diagnosis, second level diagnosis, drug, vaccine, and lot number features (lenient F-measure in the third evaluation: 0.897, 0.817, 0.858, 0.874, and 0.914, respectively). In terms of case classification, high sensitivity was achieved (83.1%); this was equal and better compared to the text classifier (83.1%) and the online tool (40.7%), respectively.
  
  
  CONCLUSIONS
  Our VaeTM implementation of a semantic text mining strategy shows promise in providing accurate and efficient extraction of key features from VAERS narratives.
authors.affiliation: Center for Biologics Evaluation and Research, Food and Drug Administration, Rockville, Maryland, USA. taxiarchis.botsis@fda.hhs.gov
authors.forename: Taxiarchis
authors.initials: T
authors.lastname: Botsis
authors.forename: Thomas
authors.initials: T
authors.lastname: Buttolph
authors.forename: Michael D
authors.initials: MD
authors.lastname: Nguyen
authors.forename: Scott
authors.initials: S
authors.lastname: Winiecki
authors.forename: Emily Jane
authors.initials: EJ
authors.lastname: Woo
authors.forename: Robert
authors.initials: R
authors.lastname: Ball
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems; D057225:Data Mining / Q000379:methods*; D006801:Humans; D009323:Natural Language Processing; D012660:Semantics; D014481:United States; D014612:Vaccines / Q000009:adverse effects*
chemical_list: D014612:Vaccines