title: Use of a support vector machine for categorizing free-text notes: assessment of accuracy across two institutions.
abstract: BACKGROUND
  Electronic health record (EHR) users must regularly review large amounts of data in order to make informed clinical decisions, and such review is time-consuming and often overwhelming. Technologies like automated summarization tools, EHR search engines and natural language processing have been shown to help clinicians manage this information.
  
  
  OBJECTIVE
  To develop a support vector machine (SVM)-based system for identifying EHR progress notes pertaining to diabetes, and to validate it at two institutions.
  
  
  METHODS
  We retrieved 2000 EHR progress notes from patients with diabetes at the Brigham and Women's Hospital (1000 for training and 1000 for testing) and another 1000 notes from the University of Texas Physicians (for validation). We manually annotated all notes and trained a SVM using a bag of words approach. We then used the SVM on the testing and validation sets and evaluated its performance with the area under the curve (AUC) and F statistics.
  
  
  RESULTS
  The model accurately identified diabetes-related notes in both the Brigham and Women's Hospital testing set (AUC=0.956, F=0.934) and the external University of Texas Faculty Physicians validation set (AUC=0.947, F=0.935).
  
  
  CONCLUSIONS
  Overall, the model we developed was quite accurate. Furthermore, it generalized, without loss of accuracy, to another institution with a different EHR and a distinct patient and provider population.
  
  
  CONCLUSIONS
  It is possible to use a SVM-based classifier to identify EHR progress notes pertaining to diabetes, and the model generalizes well.
authors.affiliation: Department of General Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA.
authors.forename: Adam
authors.initials: A
authors.lastname: Wright
authors.forename: Allison B
authors.initials: AB
authors.lastname: McCoy
authors.forename: Stanislav
authors.initials: S
authors.lastname: Henkin
authors.forename: Abhivyakti
authors.initials: A
authors.lastname: Kale
authors.forename: Dean F
authors.initials: DF
authors.lastname: Sittig
mesh_terms: D003920:Diabetes Mellitus; D057286:Electronic Health Records; D006801:Humans; D012372:ROC Curve; D057386:Search Engine; D060388:Support Vector Machine
keywords: electronic health record; natural language processing; search; support vector machine
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):540-3
references.pmid: 21846785
references.citation: Int J Med Inform. 2010 Jul;79(7):515-22
references.pmid: 20418155
references.citation: BMC Med Inform Decis Mak. 2011 May 25;11:36
references.pmid: 21612639
references.citation: J Am Med Inform Assoc. 2012 Sep-Oct;19(5):713-8
references.pmid: 22582202
references.citation: J Biomed Inform. 2011 Aug;44(4):688-99
references.pmid: 21440086
references.citation: AMIA Annu Symp Proc. 2011;2011:888-94
references.pmid: 22195147
references.citation: J Am Med Inform Assoc. 2008 Jan-Feb;15(1):14-24
references.pmid: 17947624
references.citation: Bioinformatics. 2005 Oct 15;21(20):3940-1
references.pmid: 16096348
references.citation: AMIA Annu Symp Proc. 2009 Nov 14;2009:619-23
references.pmid: 20351929
references.citation: J Biomed Inform. 2010 Dec;43(6):891-901
references.pmid: 20884377
references.citation: Arch Intern Med. 2010 Dec 13;170(22):1989-95
references.pmid: 21149756
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):539
references.pmid: 21846784
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):544-51
references.pmid: 21846786
references.citation: AMIA Annu Symp Proc. 2011;2011:189-96
references.pmid: 22195070
references.citation: Acad Med. 1999 Aug;74(8):890-5
references.pmid: 10495728
references.citation: AMIA Annu Symp Proc. 2006;:941
references.pmid: 17238560
references.citation: Yearb Med Inform. 2008;:128-44
references.pmid: 18660887
references.citation: Health Info Libr J. 2004 Jun;21(2):102-8
references.pmid: 15191601
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):568-73
references.pmid: 21724741
references.citation: AMIA Annu Symp Proc. 2003;:855
references.pmid: 14728360
references.citation: Proceedings (IEEE Int Conf Bioinformatics Biomed). 2009 Nov;2009:314-319
references.pmid: 21152268
references.citation: J Am Med Inform Assoc. 2010 Jan-Feb;17(1):49-53
references.pmid: 20064801
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):588-93
references.pmid: 21597105
references.citation: Appl Clin Inform. 2012 Feb 22;3(1):80-93
references.pmid: 22468161
references.citation: BMC Med Inform Decis Mak. 2006 Jul 26;6:30
references.pmid: 16872495
references.citation: J Biomed Inform. 2002 Aug;35(4):222-35
references.pmid: 12755517
references.citation: J Am Med Inform Assoc. 2008 Jan-Feb;15(1):25-8
references.pmid: 17947622