title: A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summaries.
abstract: OBJECTIVE
  The authors' goal was to develop and evaluate machine-learning-based approaches to extracting clinical entities-including medical problems, tests, and treatments, as well as their asserted status-from hospital discharge summaries written using natural language. This project was part of the 2010 Center of Informatics for Integrating Biology and the Bedside/Veterans Affairs (VA) natural-language-processing challenge.
  
  
  METHODS
  The authors implemented a machine-learning-based named entity recognition system for clinical text and systematically evaluated the contributions of different types of features and ML algorithms, using a training corpus of 349 annotated notes. Based on the results from training data, the authors developed a novel hybrid clinical entity extraction system, which integrated heuristic rule-based modules with the ML-base named entity recognition module. The authors applied the hybrid system to the concept extraction and assertion classification tasks in the challenge and evaluated its performance using a test data set with 477 annotated notes.
  
  
  METHODS
  Standard measures including precision, recall, and F-measure were calculated using the evaluation script provided by the Center of Informatics for Integrating Biology and the Bedside/VA challenge organizers. The overall performance for all three types of clinical entities and all six types of assertions across 477 annotated notes were considered as the primary metric in the challenge.
  
  
  CONCLUSIONS
  Systematic evaluation on the training set showed that Conditional Random Fields outperformed Support Vector Machines, and semantic information from existing natural-language-processing systems largely improved performance, although contributions from different types of features varied. The authors' hybrid entity extraction system achieved a maximum overall F-score of 0.8391 for concept extraction (ranked second) and 0.9313 for assertion classification (ranked fourth, but not statistically different than the first three systems) on the test data set in the challenge.
authors.affiliation: Department of Biomedical Informatics, Vanderbilt University, School of Medicine, Nashville, Tennessee 37232, USA.
authors.forename: Min
authors.initials: M
authors.lastname: Jiang
authors.forename: Yukun
authors.initials: Y
authors.lastname: Chen
authors.forename: Mei
authors.initials: M
authors.lastname: Liu
authors.forename: S Trent
authors.initials: ST
authors.lastname: Rosenbloom
authors.forename: Subramani
authors.initials: S
authors.lastname: Mani
authors.forename: Joshua C
authors.initials: JC
authors.lastname: Denny
authors.forename: Hua
authors.initials: H
authors.lastname: Xu
mesh_terms: D001185:Artificial Intelligence; D057225:Data Mining / Q000145:classification; D020000:Decision Support Systems, Clinical / Q000145:classification; D057286:Electronic Health Records / Q000145:classification; D006801:Humans; D009323:Natural Language Processing; D010351:Patient Discharge; D010363:Pattern Recognition, Automated; D012660:Semantics; D018875:Vocabulary, Controlled