Validating a guidelines-based asthma decision support system: step two.
OBJECTIVE
Previously we described a computerized asthma decision support system and measured its accuracy in classifying asthma severity using pulmonologists as the gold standard. It is possible that the computerized decision support system may outperform pulmonologists in classifying asthma severity. The purpose of this study was to determine whether the computerized decision support system or pulmonologists have better accuracy with the 2007 edition of the Global Initiative for Asthma guidelines in classifying asthma severity. DESIGN. One hundred asthma case scenarios were developed by a medical resident and classified according to the 2007 Global Initiative for Asthma guidelines. These case scenarios were deemed the gold standard. These case scenarios were then entered into a computerized asthma decision support system and also presented to a group of 10 practicing pulmonologists. Sensitivity, specificity, and agreement were calculated for the computerized decision support system and the pulmonologists.


RESULTS
The sensitivity and specificity of the computerized decision support system was 100%. The sensitivity of the pulmonologist was 94.2% and the specificity was 70.0%. Agreement between the computerized decision support system and the gold standard was perfect with a Kappa = 1.00. The Kappa between the pulmonologists and the gold standard was equal to 0.672.


CONCLUSIONS
The computerized decision support system outperformed pulmonologists in the task of assigning asthma severity according to the 2007 Global Initiative for Asthma guidelines.
Stern
Thomas
T
Department of Internal Medicine and the Department of Biostatistics of Carolinas Medical Center, PO Box 32861, Charlotte, NC 28232-2861, USA. Thomas.stern@carolinashealthcare.org
Hunt
Jody
J
Norton
H James
HJ
D001249:Asthma / Q000175:diagnosis*; D020000:Decision Support Systems, Clinical; D006801:Humans; D017410:Practice Guidelines as Topic; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity