title: Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus.
abstract: The 2014 i2b2/UTHealth natural language processing shared task featured a track focused on the de-identification of longitudinal medical records. For this track, we de-identified a set of 1304 longitudinal medical records describing 296 patients. This corpus was de-identified under a broad interpretation of the HIPAA guidelines using double-annotation followed by arbitration, rounds of sanity checking, and proof reading. The average token-based F1 measure for the annotators compared to the gold standard was 0.927. The resulting annotations were used both to de-identify the data and to set the gold standard for the de-identification track of the 2014 i2b2/UTHealth shared task. All annotated private health information were replaced with realistic surrogates automatically and then read over and corrected manually. The resulting corpus is the first of its kind made available for de-identification research. This corpus was first used for the 2014 i2b2/UTHealth shared task, during which the systems achieved a mean F-measure of 0.872 and a maximum F-measure of 0.964 using entity-based micro-averaged evaluations.
authors: School of Library and Information Science, Simmons College, Boston, MA, USA. Electronic address: stubbs@simmons.edu.
authors: Amber
authors: A
authors: Stubbs
authors: Department of Information Studies, State University of New York at Albany, Albany, NY, USA.
authors: Özlem
authors: Ö
authors: Uzuner
mesh_terms: D015331:Cohort Studies; D016494:Computer Security; D003219:Confidentiality; D057225:Data Mining / Q000379:methods*; D004282:Documentation / Q000379:methods; D057286:Electronic Health Records / Q000458:organization & administration*; D033262:Narration; D009323:Natural Language Processing; D010363:Pattern Recognition, Automated / Q000379:methods*; D018875:Vocabulary, Controlled
keywords: Annotation; De-identification; HIPAA; Natural language processing