title: Evaluating the state of the art in coreference resolution for electronic medical records.
abstract: BACKGROUND
  The fifth i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records conducted a systematic review on resolution of noun phrase coreference in medical records. Informatics for Integrating Biology and the Bedside (i2b2) and the Veterans Affair (VA) Consortium for Healthcare Informatics Research (CHIR) partnered to organize the coreference challenge. They provided the research community with two corpora of medical records for the development and evaluation of the coreference resolution systems. These corpora contained various record types (ie, discharge summaries, pathology reports) from multiple institutions.
  
  
  METHODS
  The coreference challenge provided the community with two annotated ground truth corpora and evaluated systems on coreference resolution in two ways: first, it evaluated systems for their ability to identify mentions of concepts and to link together those mentions. Second, it evaluated the ability of the systems to link together ground truth mentions that refer to the same entity. Twenty teams representing 29 organizations and nine countries participated in the coreference challenge.
  
  
  RESULTS
  The teams' system submissions showed that machine-learning and rule-based approaches worked best when augmented with external knowledge sources and coreference clues extracted from document structure. The systems performed better in coreference resolution when provided with ground truth mentions. Overall, the systems struggled in solving coreference resolution for cases that required domain knowledge.
authors.affiliation: Department of Information Studies, University at Albany, SUNY, Albany, New York 12222, USA. ouzuner@albany.edu
authors.forename: Ozlem
authors.initials: O
authors.lastname: Uzuner
authors.forename: Andreea
authors.initials: A
authors.lastname: Bodnari
authors.forename: Shuying
authors.initials: S
authors.lastname: Shen
authors.forename: Tyler
authors.initials: T
authors.lastname: Forbush
authors.forename: John
authors.initials: J
authors.lastname: Pestian
authors.forename: Brett R
authors.initials: BR
authors.lastname: South
mesh_terms: D057286:Electronic Health Records; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods*; D009323:Natural Language Processing