title: Normalizing acronyms and abbreviations to aid patient understanding of clinical texts: ShARe/CLEF eHealth Challenge 2013, Task 2.
abstract: BACKGROUND
  The ShARe/CLEF eHealth challenge lab aims to stimulate development of natural language processing and information retrieval technologies to aid patients in understanding their clinical reports. In clinical text, acronyms and abbreviations, also referenced as short forms, can be difficult for patients to understand. For one of three shared tasks in 2013 (Task 2), we generated a reference standard of clinical short forms normalized to the Unified Medical Language System. This reference standard can be used to improve patient understanding by linking to web sources with lay descriptions of annotated short forms or by substituting short forms with a more simplified, lay term.
  
  
  METHODS
  In this study, we evaluate 1) accuracy of participating systems' normalizing short forms compared to a majority sense baseline approach, 2) performance of participants' systems for short forms with variable majority sense distributions, and 3) report the accuracy of participating systems' normalizing shared normalized concepts between the test set and the Consumer Health Vocabulary, a vocabulary of lay medical terms.
  
  
  RESULTS
  The best systems submitted by the five participating teams performed with accuracies ranging from 43 to 72 %. A majority sense baseline approach achieved the second best performance. The performance of participating systems for normalizing short forms with two or more senses with low ambiguity (majority sense greater than 80 %) ranged from 52 to 78 % accuracy, with two or more senses with moderate ambiguity (majority sense between 50 and 80 %) ranged from 23 to 57 % accuracy, and with two or more senses with high ambiguity (majority sense less than 50 %) ranged from 2 to 45 % accuracy. With respect to the ShARe test set, 69 % of short form annotations contained common concept unique identifiers with the Consumer Health Vocabulary. For these 2594 possible annotations, the performance of participating systems ranged from 50 to 75 % accuracy.
  
  
  CONCLUSIONS
  Short form normalization continues to be a challenging problem. Short form normalization systems perform with moderate to reasonable accuracies. The Consumer Health Vocabulary could enrich its knowledge base with missed concept unique identifiers from the ShARe test set to further support patient understanding of unfamiliar medical terms.
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA. danielle.mowery@utah.edu.
authors.forename: Danielle L
authors.initials: DL
authors.lastname: Mowery
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
authors.forename: Brett R
authors.initials: BR
authors.lastname: South
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
authors.forename: Lee
authors.initials: L
authors.lastname: Christensen
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
authors.forename: Jianwei
authors.initials: J
authors.lastname: Leng
authors.affiliation: Nursing Science, University of Turku, and Turku University Hospital, Turku, Finland.
authors.forename: Laura-Maria
authors.initials: LM
authors.lastname: Peltonen
authors.affiliation: Nursing Science, University of Turku, and Turku University Hospital, Turku, Finland.
authors.forename: Sanna
authors.initials: S
authors.lastname: Salanterä
authors.affiliation: Data61, CSIRO, The Australian National University, University of Canberra, and University of Turku, Locked Bag 8001, Canberra, 2601, ACT, Australia.
authors.forename: Hanna
authors.initials: H
authors.lastname: Suominen
authors.affiliation: MedWhat.com, San Francisco, CA, USA.|University of Melbourne, Parkville, VIC, Australia.
authors.forename: David
authors.initials: D
authors.lastname: Martinez
authors.affiliation: Department of Computer and Systems Sciences (DSV), Stockholm University, Stockholm, Sweden.
authors.forename: Sumithra
authors.initials: S
authors.lastname: Velupillai
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY, USA.
authors.forename: Noémie
authors.initials: N
authors.lastname: Elhadad
authors.affiliation: Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
authors.forename: Guergana
authors.initials: G
authors.lastname: Savova
authors.affiliation: Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
authors.forename: Sameer
authors.initials: S
authors.lastname: Pradhan
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
authors.forename: Wendy W
authors.initials: WW
authors.lastname: Chapman
mesh_terms: D064229:Biological Ontologies; D006801:Humans; D009323:Natural Language Processing; D017216:Telemedicine
keywords: Abbreviations; Acronyms; Consumer health information; Natural language processing; Unified Medical Language System