title: Building an automated problem list based on natural language processing: lessons learned in the early phase of development.
abstract: Detailed problem lists that comply with JCAHO requirements are important components of electronic health records. Besides improving continuity of care electronic problem lists could serve as foundation infrastructure for clinical trial recruitment, research, biosurveillance and billing informatics modules. However, physicians rarely maintain problem lists. Our team is building a system using MetaMap and UMLS to automatically populate the problem list. We report our early results evaluating the application. Three physicians generated gold standard problem lists for 100 cardiology ambulatory progress notes. Our application had 88% sensitivity and 66% precision using a non-modified UMLS dataset. The systemâs misses concentrated in the group of ambiguous problem list entries (Chi-square=27.12 p<0.0001). In addition to the explicit entries, the notes included 10% implicit entry candidates. MetaMap and UMLS are readily applicable to automate the problem list. Ambiguity in medical documents has consequences for performance evaluation of automated systems.
authors.affiliation: University of Washington, Seattle, WA, USA.
authors.forename: Imre
authors.initials: I
authors.lastname: Solti
authors.forename: Barry
authors.initials: B
authors.lastname: Aaronson
authors.forename: Grant
authors.initials: G
authors.lastname: Fletcher
authors.forename: Magdolna
authors.initials: M
authors.lastname: Solti
authors.forename: John H
authors.initials: JH
authors.lastname: Gennari
authors.forename: Melissa
authors.initials: M
authors.lastname: Cooper
authors.forename: Tom
authors.initials: T
authors.lastname: Payne
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D016247:Information Storage and Retrieval / Q000379:methods*; D016347:Medical Records Systems, Computerized / Q000458:organization & administration*; D008501:Medical Records, Problem-Oriented; D009323:Natural Language Processing; D010363:Pattern Recognition, Automated / Q000379:methods*; D013358:Subject Headings; D014861:Washington