Automatic identification of patients eligible for a pneumonia guideline: comparing the diagnostic accuracy of two decision support models.
BACKGROUND
In busy clinical settings, physicians often do not have enough time to identify patients for specific therapeutic guidelines. As a solution, decision support systems could automatically identify eligible patients and trigger computerized guidelines for specific diseases. Applying this idea to community-acquired pneumonia (CAP), we developed a Bayesian network (BN) and an artificial neural network (ANN) for identifying patients who have CAP and are eligible for a pneumonia guideline.


OBJECTIVE
The aim of this study was to determine whether the diagnostic accuracy of these two decision support models differs in terms of identifying CAP patients.


METHODS
We trained and tested the networks with a data set of 32,662 adult patients. For each network, we (1) calculated the specificity, the positive predictive value (PPV), and the negative predictive value (NPV) at a sensitivity of 95%, and (2) determined the area under the receiver operating characteristic curve (AUC) as a measure of overall accuracy. We tested for statistical difference between the AUCs using the correlated area z statistic.


RESULTS
At a sensitivity of 95%, the respective values for specificity, PPV, and NPV were: 92.3%, 15.1%, and 99.9% for the BN, and 94.0%, 18.6%, and 99.9% for the ANN. The BN had an AUC of 0.9795 (95% CI: 0.9736, 0.9843), and the ANN had an AUC of 0.9855 (95% CI: 0.9805, 0.9894). The difference between the AUCs was statistically significant (p=0.0044).


CONCLUSIONS
The networks achieved high overall accuracies on the testing data set. Because the difference in accuracies is statistically significant but not clinically significant, both networks are equally suited to drive a guideline.
Lagor
C
C
Department of Medical Informatics, LDS Hospital/ University of Utah, Salt Lake City, Utah 84143, USA. ldclagor@ihc.com
Aronsky
D
D
Fiszman
M
M
Haug
P J
PJ
D019540:Area Under Curve; D001185:Artificial Intelligence; D001499:Bayes Theorem; D017714:Community-Acquired Infections / Q000175:diagnosis; D020000:Decision Support Systems, Clinical; D003661:Decision Support Techniques; D003936:Diagnosis, Computer-Assisted; D006801:Humans; D016571:Neural Networks, Computer; D011014:Pneumonia / Q000175:diagnosis*; D012680:Sensitivity and Specificity