title: Detecting temporal expressions in medical narratives.
abstract: BACKGROUND
  Clinical practice and epidemiological information aggregation require knowing when, how long, and in what sequence medically relevant events occur. The Temporal Awareness and Reasoning Systems for Question Interpretation (TARSQI) Toolkit (TTK) is a complete, open source software package for the temporal ordering of events within narrative text documents. TTK was developed on newspaper articles. We extended TTK to support medical notes using veterans' affairs (VA) clinical notes and compared it to TTK.
  
  
  METHODS
  We used a development set consisting of 200 VA clinical notes to modify and append rules to TTK's time tagger, creating Med-TTK. We then evaluated the performances of TTK and Med-TTK on an independent random selection of 100 clinical notes. Evaluation tasks were to identify and classify time-referring expressions as one of four temporal classes (DATE, TIME, DURATION, and SET). The reference standard for this test set was generated by dual human manual review with disagreements resolved by a third reviewer. Outcome measures included recall and precision for each class, and inter-rater agreement scores.
  
  
  RESULTS
  There were 3146 temporal expressions in the reference standard. TTK identified 1595 temporal expressions. Recall was 0.15 (95% confidence interval [CI] 0.12-0.15) and precision was 0.27 (95% CI 0.25-0.29) for TTK. Med-TTK identified 3174 expressions. Recall was 0.86 (95% CI 0.84-0.87) and precision was 0.85 (95% CI 0.84-0.86) for Med-TTK.
  
  
  CONCLUSIONS
  The algorithms for identifying and classifying temporal expressions in medical narratives developed within Med-TTK significantly improved performance compared to TTK. Natural language processing applications such as Med-TTK provide a foundation for meaningful longitudinal mapping of patient history events among electronic health records. The tool can be accessed at the following site: http://code.google.com/p/med-ttk/.
authors.affiliation: Geriatric Research Education and Clinical Center, Tennessee Valley Healthcare System, Department of Veterans Affairs, Nashville, TN, USA. ruth.reeves2@va.gov
authors.forename: Ruth M
authors.initials: RM
authors.lastname: Reeves
authors.forename: Ferdo R
authors.initials: FR
authors.lastname: Ong
authors.forename: Michael E
authors.initials: ME
authors.lastname: Matheny
authors.forename: Joshua C
authors.initials: JC
authors.lastname: Denny
authors.forename: Dominik
authors.initials: D
authors.lastname: Aronsky
authors.forename: Glenn T
authors.initials: GT
authors.lastname: Gobbel
authors.forename: Diane
authors.initials: D
authors.lastname: Montella
authors.forename: Theodore
authors.initials: T
authors.lastname: Speroff
authors.forename: Steven H
authors.initials: SH
authors.lastname: Brown
mesh_terms: D057286:Electronic Health Records / Q000706:statistics & numerical data*; D055991:Health Records, Personal; D033262:Narration; D009323:Natural Language Processing; D010363:Pattern Recognition, Automated / Q000379:methods*; D012984:Software; D013997:Time Factors; D014481:United States; D018875:Vocabulary, Controlled