Randomized controlled trial of an automated problem list with improved sensitivity.
OBJECTIVE
To improve the completeness and timeliness of an electronic problem list, we have developed a system using Natural Language Processing (NLP) to automatically extract potential medical problems from clinical, free-text documents; these problems are then proposed for inclusion in an electronic problem list management application.


METHODS
A prospective randomized controlled evaluation of the Automatic Problem List (APL) system in an intensive care unit and in a cardiovascular surgery unit is reported here. A total of 247 patients were enrolled: 76 in an initial control phase and 171 in the randomized controlled trial that followed. During this latter phase, patients were randomly assigned to a control or an intervention group. All patients had their documents analyzed by the system, but the medical problems discovered were only proposed in the problem list for intervention patients. We measured the sensitivity, specificity, positive and negative predictive values, likelihood ratios and the timeliness of the problem lists.


RESULTS
Our system significantly increased the sensitivity of the problem lists in the intensive care unit, from about 9% to 41%, and even 77% if problems automatically proposed but not acknowledged by users were also considered. Timeliness of addition of problems to the list was greatly improved, with a time between a problem's first mention in a clinical document and its addition to the problem list reduced from about 6 days to less than 2 days. No significant effect was observed in the cardiovascular surgery unit.
Meystre
Stéphane M
SM
Department of Biomedical Informatics, University of Utah, School of Medicine, Salt Lake City, UT 84112-5750, USA. stephane.meystre@hsc.utah.edu
Haug
Peter J
PJ
D006801:Humans; D007362:Intensive Care Units; D016347:Medical Records Systems, Computerized; D008501:Medical Records, Problem-Oriented; D009323:Natural Language Processing; D011446:Prospective Studies; D012680:Sensitivity and Specificity; D014589:Utah