title: Building gold standard corpora for medical natural language processing tasks.
abstract: We present the construction of three annotated corpora to serve as gold standards for medical natural language processing (NLP) tasks. Clinical notes from the medical record, clinical trial announcements, and FDA drug labels are annotated. We report high inter-annotator agreements (overall F-measures between 0.8467 and 0.9176) for the annotation of Personal Health Information (PHI) elements for a de-identification task and of medications, diseases/disorders, and signs/symptoms for information extraction (IE) task. The annotated corpora of clinical trials and FDA labels will be publicly released and to facilitate translational NLP tasks that require cross-corpora interoperability (e.g. clinical trial eligibility screening) their annotation schemas are aligned with a large scale, NIH-funded clinical text annotation project.
authors.affiliation: Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.
authors.forename: Louise
authors.initials: L
authors.lastname: Deleger
authors.forename: Qi
authors.initials: Q
authors.lastname: Li
authors.forename: Todd
authors.initials: T
authors.lastname: Lingren
authors.forename: Megan
authors.initials: M
authors.lastname: Kaiser
authors.forename: Katalin
authors.initials: K
authors.lastname: Molnar
authors.forename: Laura
authors.initials: L
authors.lastname: Stoutenborough
authors.forename: Michal
authors.initials: M
authors.lastname: Kouril
authors.forename: Keith
authors.initials: K
authors.lastname: Marsolo
authors.forename: Imre
authors.initials: I
authors.lastname: Solti
mesh_terms: D002986:Clinical Trials as Topic; D004348:Drug Labeling; D008499:Medical Records; D009323:Natural Language Processing; D012984:Software; D014481:United States; D014486:United States Food and Drug Administration