eQuality: electronic quality assessment from narrative clinical reports.
OBJECTIVE
To evaluate an electronic quality (eQuality) assessment tool for dictated disability examination records.


METHODS
We applied automated concept-based indexing techniques to automated quality screening of Department of Veterans Affairs spine disability examinations that had previously undergone gold standard quality review by human experts using established quality indicators. We developed automated quality screening rules and refined them iteratively on a training set of disability examination reports. We applied the resulting rules to a novel test set of spine disability examination reports. The initial data set was composed of all electronically available examination reports (N=125,576) finalized by the Veterans Health Administration between July and September 2001.


RESULTS
Sensitivity was 91% for the training set and 87% for the test set (P-.02). Specificity was 74% for the training set and 71% for the test set (P=.44). Human performance ranged from 4% to 6% higher (P<.001) than the eQuality tool in sensitivity and 13% to 16% higher in specificity (P<.001). In addition, the eQuality tool was equivalent or higher in sensitivity for 5 of 9 individual quality indicators.


CONCLUSIONS
The results demonstrate that a properly authored computer-based expert systems approach can perform quality measurement as well as human reviewers for many quality indicators. Although automation will likely always rely on expert guidance to be accurate and meaningful, eQuality is an important new method to assist clinicians in their efforts to practice safe and effective medicine.
Brown
Steven H
SH
Department of Veterans Affairs Compensation and Pension Examination Program, Nashville, Tenn, USA. steven.brown@med.va.gov
Speroff
Theodore
T
Fielstein
Elliot M
EM
Bauer
Brent A
BA
Wahner-Roedler
Dietlind L
DL
Greevy
Robert
R
Elkin
Peter L
PL
D000465:Algorithms; D004185:Disability Evaluation; D006801:Humans; D016347:Medical Records Systems, Computerized / Q000592:standards*; D011785:Quality Assurance, Health Care / Q000379:methods*; D012189:Retrospective Studies; D012680:Sensitivity and Specificity; D013122:Spinal Diseases / Q000534:rehabilitation*; D014481:United States; D014493:United States Department of Veterans Affairs / Q000706:statistics & numerical data