title: De-identification of clinical narratives through writing complexity measures.
abstract: OBJECTIVE
  Electronic health records contain a substantial quantity of clinical narrative, which is increasingly reused for research purposes. To share data on a large scale and respect privacy, it is critical to remove patient identifiers. De-identification tools based on machine learning have been proposed; however, model training is usually based on either a random group of documents or a pre-existing document type designation (e.g., discharge summary). This work investigates if inherent features, such as the writing complexity, can identify document subsets to enhance de-identification performance.
  
  
  METHODS
  We applied an unsupervised clustering method to group two corpora based on writing complexity measures: a collection of over 4500 documents of varying document types (e.g., discharge summaries, history and physical reports, and radiology reports) from Vanderbilt University Medical Center (VUMC) and the publicly available i2b2 corpus of 889 discharge summaries. We compare the performance (via recall, precision, and F-measure) of de-identification models trained on such clusters with models trained on documents grouped randomly or VUMC document type.
  
  
  RESULTS
  For the Vanderbilt dataset, it was observed that training and testing de-identification models on the same stylometric cluster (with the average F-measure of 0.917) tended to outperform models based on clusters of random documents (with an average F-measure of 0.881). It was further observed that increasing the size of a training subset sampled from a specific cluster could yield improved results (e.g., for subsets from a certain stylometric cluster, the F-measure raised from 0.743 to 0.841 when training size increased from 10 to 50 documents, and the F-measure reached 0.901 when the size of the training subset reached 200 documents). For the i2b2 dataset, training and testing on the same clusters based on complexity measures (average F-score 0.966) did not significantly surpass randomly selected clusters (average F-score 0.965).
  
  
  CONCLUSIONS
  Our findings illustrate that, in environments consisting of a variety of clinical documentation, de-identification models trained on writing complexity measures are better than models trained on random groups and, in many instances, document types.
authors.affiliation: Department of Electrical Engineering & Computer Science, Vanderbilt University, Nashville, TN, United States. Electronic address: muqun.li@vanderbilt.edu.
authors.forename: Muqun
authors.initials: M
authors.lastname: Li
authors.affiliation: Group Health Research Institute, Seattle, WA, United States.
authors.forename: David
authors.initials: D
authors.lastname: Carrell
authors.affiliation: The MITRE Corporation, Bedford, MA, United States.
authors.forename: John
authors.initials: J
authors.lastname: Aberdeen
authors.affiliation: The MITRE Corporation, Bedford, MA, United States.
authors.forename: Lynette
authors.initials: L
authors.lastname: Hirschman
authors.affiliation: Department of Electrical Engineering & Computer Science, Vanderbilt University, Nashville, TN, United States; Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, United States.
authors.forename: Bradley A
authors.initials: BA
authors.lastname: Malin
mesh_terms: D016000:Cluster Analysis; D057286:Electronic Health Records; D033262:Narration; D014956:Writing
keywords: Electronic medical records; Natural language processing; Privacy
references.citation: Sci Transl Med. 2011 Apr 20;3(79):79re1
references.pmid: 21508311
references.citation: J Am Med Inform Assoc. 2013 Mar-Apr;20(2):342-8
references.pmid: 22771529
references.citation: BMC Med Res Methodol. 2010 Aug 02;10:70
references.pmid: 20678228
references.citation: J Am Med Inform Assoc. 2013 Jan 1;20(1):84-94
references.pmid: 22859645
references.citation: Clin Pharmacol Ther. 2008 Sep;84(3):362-9
references.pmid: 18500243
references.citation: AMIA Annu Symp Proc. 2008 Nov 06;:753-7
references.pmid: 18999284
references.citation: Int J Med Inform. 2010 Dec;79(12):849-59
references.pmid: 20951082
references.citation: Artif Intell Med. 2008 Jan;42(1):13-35
references.pmid: 18053696
references.citation: Proc AMIA Symp. 2002;:757-61
references.pmid: 12463926
references.citation: AMIA Annu Symp Proc. 2003;:1065
references.pmid: 14728568
references.citation: J Am Med Inform Assoc. 2013 Jan 1;20(1):77-83
references.pmid: 22947391
references.citation: Bioinformatics. 2005 Aug 1;21(15):3201-12
references.pmid: 15914541
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):540-3
references.pmid: 21846785
references.citation: Proc AMIA Symp. 2001;:17-21
references.pmid: 11825149
references.citation: J Am Med Inform Assoc. 2012 Mar-Apr;19(2):225-34
references.pmid: 22319176
references.citation: Health Care Law Mon. 2003 Jan;:13-20
references.pmid: 12587256
references.citation: Nucleic Acids Res. 2004 Jan 1;32(Database issue):D267-70
references.pmid: 14681409
references.citation: Yearb Med Inform. 2008;:128-44
references.pmid: 18660887
references.citation: Am J Clin Pathol. 2004 Feb;121(2):176-86
references.pmid: 14983930
references.citation: Am J Manag Care. 2011 Dec;17(12 Spec No.):SP117-24
references.pmid: 22216770
references.citation: J Am Med Inform Assoc. 2007 Sep-Oct;14(5):574-80
references.pmid: 17823086
references.citation: J Am Med Inform Assoc. 2007 Sep-Oct;14(5):550-63
references.pmid: 17600094
references.citation: J Biomed Inform. 2009 Oct;42(5):760-72
references.pmid: 19683066
references.citation: BMC Bioinformatics. 2008 Nov 19;9 Suppl 11:S3
references.pmid: 19025689
references.citation: J Appl Psychol. 1948 Jun;32(3):221-33
references.pmid: 18867058
references.citation: Behav Res Methods. 2010 May;42(2):381-92
references.pmid: 20479170
references.citation: J Am Med Inform Assoc. 2011 Mar-Apr;18(2):181-6
references.pmid: 21233086
references.citation: J Am Med Inform Assoc. 2007 Sep-Oct;14(5):564-73
references.pmid: 17600096
references.citation: Methods Inf Med. 2006;45(3):246-52
references.pmid: 16685332
references.citation: J Am Med Inform Assoc. 2008 Jan-Feb;15(1):54-64
references.pmid: 17947628