title: Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010.
abstract: OBJECTIVE
  As clinical text mining continues to mature, its potential as an enabling technology for innovations in patient care and clinical research is becoming a reality. A critical part of that process is rigid benchmark testing of natural language processing methods on realistic clinical narrative. In this paper, the authors describe the design and performance of three state-of-the-art text-mining applications from the National Research Council of Canada on evaluations within the 2010 i2b2 challenge.
  
  
  METHODS
  The three systems perform three key steps in clinical information extraction: (1) extraction of medical problems, tests, and treatments, from discharge summaries and progress notes; (2) classification of assertions made on the medical problems; (3) classification of relations between medical concepts. Machine learning systems performed these tasks using large-dimensional bags of features, as derived from both the text itself and from external sources: UMLS, cTAKES, and Medline.
  
  
  METHODS
  Performance was measured per subtask, using micro-averaged F-scores, as calculated by comparing system annotations with ground-truth annotations on a test set.
  
  
  RESULTS
  The systems ranked high among all submitted systems in the competition, with the following F-scores: concept extraction 0.8523 (ranked first); assertion detection 0.9362 (ranked first); relationship detection 0.7313 (ranked second).
  
  
  CONCLUSIONS
  For all tasks, we found that the introduction of a wide range of features was crucial to success. Importantly, our choice of machine learning algorithms allowed us to be versatile in our feature design, and to introduce a large number of features without overfitting and without encountering computing-resource bottlenecks.
authors.affiliation: Institute for Information Technology, National Research Council, Ottawa, Ontario, Canada. berry.debruijn@nrc-cnrc.gc.ca
authors.forename: Berry
authors.initials: B
authors.lastname: de Bruijn
authors.forename: Colin
authors.initials: C
authors.lastname: Cherry
authors.forename: Svetlana
authors.initials: S
authors.lastname: Kiritchenko
authors.forename: Joel
authors.initials: J
authors.lastname: Martin
authors.forename: Xiaodan
authors.initials: X
authors.lastname: Zhu
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D019985:Benchmarking; D002170:Canada; D057225:Data Mining / Q000145:classification; D057286:Electronic Health Records / Q000145:classification; D006801:Humans; D009323:Natural Language Processing