Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography.
OBJECTIVE
To investigate the effects of incorrect computer output on the reliability of the decisions of human users. This work followed an independent UK clinical trial that evaluated the impact of computer-aided detection(CAD) in breast screening. The aim was to use data from this trial to feed into probabilistic models (similar to those used in "reliability engineering") which would detect and assess possible ways of improving the human-CAD interaction. Some analyses required extra data; therefore, two supplementary studies were conducted. Study 1 was designed to elucidate the effects of computer failure on human performance. Study 2 was conducted to clarify unexpected findings from Study 1.


METHODS
In Study 1, 20 film readers viewed 60 sets of mammograms (30 of which contained cancer) and provided "recall/no recall" decisions for each case. Computer output for each case was available to the participants. The test set was designed to contain an unusually large proportion (50%) of cancers for which CAD had generated incorrect output. In Study 2, 19 different readers viewed the same set of cases in similar conditions except that computer output was not available.


RESULTS
The average sensitivity of readers in Study 1 (with CAD) was significantly lower than the average sensitivity of read-ers in Study 2 (without CAD). The difference was most marked for cancers for which CAD failed to provide correct prompting.


CONCLUSIONS
Possible automation bias effects in CAD use deserve further study because they may degrade human decision-making for some categories of cases under certain conditions. This possibility should be taken into account in the assessment and design of CAD tools.
Alberdi
Eugenio
E
Centre for Software Reliability and the Department of Psychology, City University, Northampton Square, London, UK. e.alberdi@csr.city.ac.uk
Povykalo
Andrey
A
Strigini
Lorenzo
L
Ayton
Peter
P
D000704:Analysis of Variance; D015982:Bias; D001943:Breast Neoplasms / Q000000981:diagnostic imaging; D003657:Decision Making; D003936:Diagnosis, Computer-Assisted; D005188:False Negative Reactions; D005260:Female; D005500:Follow-Up Studies; D006801:Humans; D007091:Image Processing, Computer-Assisted; D008327:Mammography; D008403:Mass Screening; D015233:Models, Statistical; D011336:Probability; D011857:Radiographic Image Interpretation, Computer-Assisted; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity