title: Automated outcome classification of emergency department computed tomography imaging reports.
abstract: BACKGROUND
  Reliably abstracting outcomes from free-text electronic health records remains a challenge. While automated classification of free text has been a popular medical informatics topic, performance validation using real-world clinical data has been limited. The two main approaches are linguistic (natural language processing [NLP]) and statistical (machine learning). The authors have developed a hybrid system for abstracting computed tomography (CT) reports for specified outcomes.
  
  
  OBJECTIVE
  The objective was to measure performance of a hybrid NLP and machine learning system for automated outcome classification of emergency department (ED) CT imaging reports. The hypothesis was that such a system is comparable to medical personnel doing the data abstraction.
  
  
  METHODS
  A secondary analysis was performed on a prior diagnostic imaging study on 3,710 blunt facial trauma victims. Staff radiologists dictated CT reports as free text, which were then deidentified. A trained data abstractor manually coded the reference standard outcome of acute orbital fracture, with a random subset double-coded for reliability. The data set was randomly split evenly into training and testing sets. Training patient reports were used as input to the Medical Language Extraction and Encoding (MedLEE) NLP tool to create structured output containing standardized medical terms and modifiers for certainty and temporal status. Findings were filtered for low certainty and past/future modifiers and then combined with the manual reference standard to generate decision tree classifiers using data mining tools Waikato Environment for Knowledge Analysis (WEKA) 3.7.5 and Salford Predictive Miner 6.6. Performance of decision tree classifiers was evaluated on the testing set with or without NLP processing.
  
  
  RESULTS
  The performance of machine learning alone was comparable to prior NLP studies (sensitivity = 0.92, specificity = 0.93, precision = 0.95, recall = 0.93, f-score = 0.94), and the combined use of NLP and machine learning showed further improvement (sensitivity = 0.93, specificity = 0.97, precision = 0.97, recall = 0.96, f-score = 0.97). This performance is similar to, or better than, that of medical personnel in previous studies.
  
  
  CONCLUSIONS
  A hybrid NLP and machine learning automated classification system shows promise in coding free-text electronic clinical data.
authors.affiliation: Department of Emergency Medicine, The George Washington University, Washington, DC.
authors.forename: Kabir
authors.initials: K
authors.lastname: Yadav
authors.forename: Efsun
authors.initials: E
authors.lastname: Sarioglu
authors.forename: Meaghan
authors.initials: M
authors.lastname: Smith
authors.forename: Hyeong-Ah
authors.initials: HA
authors.lastname: Choi
mesh_terms: D001185:Artificial Intelligence; D015331:Cohort Studies; D057286:Electronic Health Records; D004632:Emergency Medical Services / Q000145:classification*; D004636:Emergency Service, Hospital / Q000145:classification*; D006801:Humans; D009323:Natural Language Processing; D012189:Retrospective Studies; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed / Q000145:classification*
references.citation: J Am Med Inform Assoc. 2010 Nov-Dec;17(6):646-51
references.pmid: 20962126
references.citation: Radiology. 2002 Jul;224(1):157-63
references.pmid: 12091676
references.citation: Radiology. 2005 Feb;234(2):323-9
references.pmid: 15591435
references.citation: J Biomed Inform. 2005 Aug;38(4):314-21
references.pmid: 16084473
references.citation: J Biomed Inform. 2001 Feb;34(1):4-14
references.pmid: 11376542
references.citation: Perspect Health Inf Manag. 2010 Oct 01;7:1a
references.pmid: 21063542
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):588-93
references.pmid: 21597105
references.citation: Ann Intern Med. 1995 May 1;122(9):681-8
references.pmid: 7702231
references.citation: Comput Biomed Res. 2000 Feb;33(1):1-10
references.pmid: 10772780
references.citation: J Trauma Acute Care Surg. 2012 Nov;73(5):1313-8
references.pmid: 22922967
references.citation: J Am Med Inform Assoc. 2006 Nov-Dec;13(6):691-5
references.pmid: 16929043
references.citation: Nucleic Acids Res. 2004 Jan 1;32(Database issue):D267-70
references.pmid: 14681409
references.citation: Yearb Med Inform. 2008;:128-44
references.pmid: 18660887
references.citation: Proc AMIA Symp. 1999;:216-20
references.pmid: 10566352
references.citation: Methods Inf Med. 1998 Jan;37(1):1-7
references.pmid: 9550840
references.citation: J Am Med Inform Assoc. 2010 Sep-Oct;17(5):507-13
references.pmid: 20819853
references.citation: J Am Med Inform Assoc. 2000 Nov-Dec;7(6):593-604
references.pmid: 11062233
references.citation: J Am Med Inform Assoc. 2012 Sep-Oct;19(5):913-6
references.pmid: 22291166
references.citation: Proceedings (IEEE Int Conf Bioinformatics Biomed). 2009 Nov;2009:314-319
references.pmid: 21152268
references.citation: Proc AMIA Symp. 2000;:270-4
references.pmid: 11079887
references.citation: J Am Med Inform Assoc. 1994 Mar-Apr;1(2):161-74
references.pmid: 7719797