title: Scaling-up NLP Pipelines to Process Large Corpora of Clinical Notes.
abstract: BACKGROUND
  This article is part of the Focus Theme of Methods of Information in Medicine on "Big Data and Analytics in Healthcare".
  
  
  OBJECTIVE
  This paper describes the scale-up efforts at the VA Salt Lake City Health Care System to address processing large corpora of clinical notes through a natural language processing (NLP) pipeline. The use case described is a current project focused on detecting the presence of an indwelling urinary catheter in hospitalized patients and subsequent catheter-associated urinary tract infections.
  
  
  METHODS
  An NLP algorithm using v3NLP was developed to detect the presence of an indwelling urinary catheter in hospitalized patients. The algorithm was tested on a small corpus of notes on patients for whom the presence or absence of a catheter was already known (reference standard). In planning for a scale-up, we estimated that the original algorithm would have taken 2.4 days to run on a larger corpus of notes for this project (550,000 notes), and 27 days for a corpus of 6 million records representative of a national sample of notes. We approached scaling-up NLP pipelines through three techniques: pipeline replication via multi-threading, intra-annotator threading for tasks that can be further decomposed, and remote annotator services which enable annotator scale-out.
  
  
  RESULTS
  The scale-up resulted in reducing the average time to process a record from 206 milliseconds to 17 milliseconds or a 12- fold increase in performance when applied to a corpus of 550,000 notes.
  
  
  CONCLUSIONS
  Purposely simplistic in nature, these scale-up efforts are the straight forward evolution from small scale NLP processing to larger scale extraction without incurring associated complexities that are inherited by the use of the underlying UIMA framework. These efforts represent generalizable and widely applicable techniques that will aid other computationally complex NLP pipelines that are of need to be scaled out for processing and analyzing big data.
authors.affiliation: Guy Divita, University of Utah School of Medicine, Division of Epidemiology, 295 Chipeta Way, Salt Lake City, UT 84132, USA, E-mail: guy.divita@hsc.utah.edu.
authors.forename: G
authors.initials: G
authors.lastname: Divita
authors.forename: M
authors.initials: M
authors.lastname: Carter
authors.forename: A
authors.initials: A
authors.lastname: Redd
authors.forename: Q
authors.initials: Q
authors.lastname: Zeng
authors.forename: K
authors.initials: K
authors.lastname: Gupta
authors.forename: B
authors.initials: B
authors.lastname: Trautner
authors.forename: M
authors.initials: M
authors.lastname: Samore
authors.forename: A
authors.initials: A
authors.lastname: Gundlapalli
mesh_terms: D000465:Algorithms; D055499:Catheter-Related Infections / Q000175:diagnosis / Q000453:epidemiology* / Q000517:prevention & control; D057225:Data Mining / Q000379:methods; D066264:Datasets as Topic / Q000706:statistics & numerical data; D020000:Decision Support Systems, Clinical / Q000458:organization & administration; D057286:Electronic Health Records / Q000145:classification* / Q000706:statistics & numerical data*; D006787:Hospitals, Veterans / Q000706:statistics & numerical data; D006801:Humans; D000069550:Machine Learning; D009323:Natural Language Processing; D015995:Prevalence; D018570:Risk Assessment / Q000379:methods; D014546:Urinary Catheterization / Q000706:statistics & numerical data*; D014552:Urinary Tract Infections / Q000453:epidemiology* / Q000517:prevention & control; D014589:Utah / Q000453:epidemiology; D018875:Vocabulary, Controlled
keywords: Natural language processing; big data; scale-up