title: Assessing the feasibility of large-scale natural language processing in a corpus of ordinary medical records: a lexical analysis.
abstract: OBJECTIVE
  Identify the lexical content of a large corpus of ordinary medical records to assess the feasibility of large-scale natural language processing.
  
  
  METHODS
  A corpus of 560 megabytes of medical record text from an academic medical center was broken into individual words and compared with the words in six medical vocabularies, a common word list, and a database of patient names. Unrecognized words were assessed for algorithmic and contextual approaches to identifying more words, while the remainder were analyzed for spelling correctness.
  
  
  RESULTS
  About 60% of the words occurred in the medical vocabularies, common word list, or names database. Of the remainder, one-third were recognizable by other means. Of the remaining unrecognizable words, over three-fourths represented correctly spelled real words and the rest were misspellings.
  
  
  CONCLUSIONS
  Large-scale generalized natural language processing methods for the medical record will require expansion of existing vocabularies, spelling error correction, and other algorithmic approaches to map words into those from clinical vocabularies.
authors.affiliation: Division of Medical Informatics and Outcomes Research, Oregon Health Sciences University, USA.
authors.forename: W R
authors.initials: WR
authors.lastname: Hersh
authors.forename: E M
authors.initials: EM
authors.lastname: Campbell
authors.forename: S E
authors.initials: SE
authors.lastname: Malveau
mesh_terms: D000465:Algorithms; D016347:Medical Records Systems, Computerized; D009323:Natural Language Processing; D017432:Unified Medical Language System; D018875:Vocabulary, Controlled