Evaluation of LabRespond, a new automated validation system for clinical laboratory test results.
BACKGROUND
Manual validation of laboratory test results is time-consuming, creating a demand for expert systems to automate this process. We have started to set up the program "LabRespond", which covers five validation levels: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation system based on statistical methods, in contrast to the commercially available program "VALAB", a rule-based automated validation system.


METHODS
In the present study, 163 willfully altered, erroneous test results out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) who validated these test results manually. The test results rejected by three or more clinical chemists (n = 281) served as a secondary reference standard.


RESULTS
The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, respectively (difference not significant). The false-positive rates were 82.7% for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Using the consensus of three or more clinical chemists as the secondary reference standard, we found error recovery rates of 64.8% for LabRespond and 72.2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal.


CONCLUSIONS
The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate than the clinical chemists participating in this study, and a performance comparable to VALAB.
Oosterhuis
W P
WP
Department of Clinical Chemistry and Hematology, St. Elisabeth Hospital, Hilvarenbeekseweg 60, 5022 GC Tilburg, The Netherlands. oosthuis@knmg.nl
Ulenkate
H J
HJ
Goldschmidt
H M
HM
D019411:Clinical Laboratory Techniques; D003627:Data Interpretation, Statistical; D005103:Expert Systems; D006801:Humans; D011786:Quality Control