title: Multilayered temporal modeling for the clinical domain.
abstract: OBJECTIVE
  To develop an open-source temporal relation discovery system for the clinical domain. The system is capable of automatically inferring temporal relations between events and time expressions using a multilayered modeling strategy. It can operate at different levels of granularity--from rough temporality expressed as event relations to the document creation time (DCT) to temporal containment to fine-grained classic Allen-style relations.
  
  
  METHODS
  We evaluated our systems on 2 clinical corpora. One is a subset of the Temporal Histories of Your Medical Events (THYME) corpus, which was used in SemEval 2015 Task 6: Clinical TempEval. The other is the 2012 Informatics for Integrating Biology and the Bedside (i2b2) challenge corpus. We designed multiple supervised machine learning models to compute the DCT relation and within-sentence temporal relations. For the i2b2 data, we also developed models and rule-based methods to recognize cross-sentence temporal relations. We used the official evaluation scripts of both challenges to make our results comparable with results of other participating systems. In addition, we conducted a feature ablation study to find out the contribution of various features to the system's performance.
  
  
  RESULTS
  Our system achieved state-of-the-art performance on the Clinical TempEval corpus and was on par with the best systems on the i2b2 2012 corpus. Particularly, on the Clinical TempEval corpus, our system established a new F1 score benchmark, statistically significant as compared to the baseline and the best participating system.
  
  
  CONCLUSIONS
  Presented here is the first open-source clinical temporal relation discovery system. It was built using a multilayered temporal modeling strategy and achieved top performance in 2 major shared tasks.
authors.affiliation: Boston Children's Hospital Boston, Boston, Massachusetts, USA chen.lin@childrens.harvard.edu.
authors.forename: Chen
authors.initials: C
authors.lastname: Lin
authors.affiliation: Boston Children's Hospital Boston, Boston, Massachusetts, USA Harvard Medical School, Harvard University, Boston, Massachusetts, USA.
authors.forename: Dmitriy
authors.initials: D
authors.lastname: Dligach
authors.affiliation: Boston Children's Hospital Boston, Boston, Massachusetts, USA Harvard Medical School, Harvard University, Boston, Massachusetts, USA.
authors.forename: Timothy A
authors.initials: TA
authors.lastname: Miller
authors.affiliation: Department of Computer and Information Sciences, University of Alabama at Birmingham, Birmingham, Alabama, USA.
authors.forename: Steven
authors.initials: S
authors.lastname: Bethard
authors.affiliation: Boston Children's Hospital Boston, Boston, Massachusetts, USA Harvard Medical School, Harvard University, Boston, Massachusetts, USA.
authors.forename: Guergana K
authors.initials: GK
authors.lastname: Savova
mesh_terms: D000465:Algorithms; D057286:Electronic Health Records; D016247:Information Storage and Retrieval / Q000379:methods*; D009323:Natural Language Processing; D010067:Ownership; D013995:Time
keywords: Allen's temporal interval relations; document creation time; electronic medical record; narrative container; natural language processing; temporal relation discovery