title: Performance of a Natural Language Processing (NLP) Tool to Extract Pulmonary Function Test (PFT) Reports from Structured and Semistructured Veteran Affairs (VA) Data.
abstract: OBJECTIVE
  Pulmonary function tests (PFTs) are objective estimates of lung function, but are not reliably stored within the Veteran Health Affairs data systems as structured data. The aim of this study was to validate the natural language processing (NLP) tool we developed-which extracts spirometric values and responses to bronchodilator administration-against expert review, and to estimate the number of additional spirometric tests identified beyond the structured data.
  
  
  METHODS
  All patients at seven Veteran Affairs Medical Centers with a diagnostic code for asthma Jan 1, 2006-Dec 31, 2012 were included. Evidence of spirometry with a bronchodilator challenge (BDC) was extracted from structured data as well as clinical documents. NLP's performance was compared against a human reference standard using a random sample of 1,001 documents.
  
  
  RESULTS
  In the validation set NLP demonstrated a precision of 98.9 percent (95 percent confidence intervals (CI): 93.9 percent, 99.7 percent), recall of 97.8 percent (95 percent CI: 92.2 percent, 99.7 percent), and an F-measure of 98.3 percent for the forced vital capacity pre- and post pairs and precision of 100 percent (95 percent CI: 96.6 percent, 100 percent), recall of 100 percent (95 percent CI: 96.6 percent, 100 percent), and an F-measure of 100 percent for the forced expiratory volume in one second pre- and post pairs for bronchodilator administration. Application of the NLP increased the proportion identified with complete bronchodilator challenge by 25 percent.
  
  
  CONCLUSIONS
  This technology can improve identification of PFTs for epidemiologic research. Caution must be taken in assuming that a single domain of clinical data can completely capture the scope of a disease, treatment, or clinical test.
authors.affiliation: Salt Lake IDEAS Center, Veteran Affairs; Division of Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah.
authors.forename: Brian C
authors.initials: BC
authors.lastname: Sauer
authors.affiliation: Salt Lake IDEAS Center, Veteran Affairs; Division of Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah.
authors.forename: Barbara E
authors.initials: BE
authors.lastname: Jones
authors.affiliation: Amgen Inc.
authors.forename: Gary
authors.initials: G
authors.lastname: Globe
authors.affiliation: Division of Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah.
authors.forename: Jianwei
authors.initials: J
authors.lastname: Leng
authors.affiliation: Division of Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah.
authors.forename: Chao-Chin
authors.initials: CC
authors.lastname: Lu
authors.affiliation: Division of Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah.
authors.forename: Tao
authors.initials: T
authors.lastname: He
authors.affiliation: Division of Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah.
authors.forename: Chia-Chen
authors.initials: CC
authors.lastname: Teng
authors.affiliation: Department of Pharmacy Practice, School of Pharmacy, Regis University.
authors.forename: Patrick
authors.initials: P
authors.lastname: Sullivan
authors.affiliation: Salt Lake IDEAS Center, Veteran Affairs; Department of Biomedical Informatics, School of Medicine, University of Utah.
authors.forename: Qing
authors.initials: Q
authors.lastname: Zeng
keywords: Data Reuse; Data Use and Quality; Electronic Health Record (EHR); Informatics; Natural Language Processing; Outcomes Assessment; Pulmonary Disease; asthma; bronchodilator challenge; natural language processing; pulmonary function