title: Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system.
abstract: OBJECTIVE
  As the US Food and Drug Administration (FDA) receives over a million adverse event reports associated with medication use every year, a system is needed to aid FDA safety evaluators in identifying reports most likely to demonstrate causal relationships to the suspect medications. We combined text mining with machine learning to construct and evaluate such a system to identify medication-related adverse event reports.
  
  
  METHODS
  FDA safety evaluators assessed 326 reports for medication-related causality. We engineered features from these reports and constructed random forest, L1 regularized logistic regression, and support vector machine models. We evaluated model accuracy and further assessed utility by generating report rankings that represented a prioritized report review process.
  
  
  RESULTS
  Our random forest model showed the best performance in report ranking and accuracy, with an area under the receiver operating characteristic curve of 0.66. The generated report ordering assigns reports with a higher probability of medication-related causality a higher rank and is significantly correlated to a perfect report ordering, with a Kendall's tau of 0.24 ( P  = .002).
  
  
  CONCLUSIONS
  Our models produced prioritized report orderings that enable FDA safety evaluators to focus on reports that are more likely to contain valuable medication-related adverse event information. Applying our models to all FDA adverse event reports has the potential to streamline the manual review process and greatly reduce reviewer workload.
authors.affiliation: Biomedical Informatics Training Program, Stanford University, Stanford, CA, USA.
authors.forename: Lichy
authors.initials: L
authors.lastname: Han
authors.affiliation: Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA.
authors.forename: Robert
authors.initials: R
authors.lastname: Ball
authors.affiliation: Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA.
authors.forename: Carol A
authors.initials: CA
authors.lastname: Pamer
authors.affiliation: Department of Genetics, Stanford University.|Department of Bioengineering, Stanford University.
authors.forename: Russ B
authors.initials: RB
authors.lastname: Altman
authors.affiliation: Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA.
authors.forename: Scott
authors.initials: S
authors.lastname: Proestel
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems; D057225:Data Mining; D064420:Drug-Related Side Effects and Adverse Reactions; D016015:Logistic Models; D000069550:Machine Learning; D008962:Models, Theoretical; D009323:Natural Language Processing; D012372:ROC Curve; D060388:Support Vector Machine; D014481:United States; D014486:United States Food and Drug Administration
keywords: drug-related side effects and adverse reactions; supervised machine learning