Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiology.
OBJECTIVE
The purpose of this study was to test whether the satisfaction of search (SOS) effect in chest radiology could be demonstrated with proper receiver operating characteristic (ROC) curves and with joint detection and localization ROC curves.


METHODS
Data from an earlier ROC study of SOS in chest radiology were analyzed with three proper ROC models and one ROC model for joint detection and localization. Fits of the models were compared on the basis of likelihood-ratio chi-squared statistics (G2). To examine further the validity of the SOS effect in chest radiology, the authors also replicated the earlier study with a new sample of readers, analyzing the new data with the same methods.


RESULTS
The proper contaminated binormal model fit the data better than the other two proper ROC models. Contaminated binormal analysis of the earlier and the replication experiment demonstrated an SOS effect: a reduction in area under the ROC curve for detection of the native abnormalities with the addition of nodules. Similarly, joint ROC analysis producing curves that appropriately cross the chance line gave similar results.


CONCLUSIONS
Preventing inappropriate chance line crossing reduces measurement error and provides more powerful statistical tests. Results of both experiments showed that the SOS effect in chest radiology can be demonstrated with ROC methods that avoid inappropriate crossing of the chance line.
Berbaum
K S
KS
Department of Radiology, University of Iowa, Iowa City 52242, USA.
Dorfman
D D
DD
Franken
E A
EA
Caldwell
R T
RT
D016009:Chi-Square Distribution; D006801:Humans; D016013:Likelihood Functions; D008175:Lung Neoplasms / Q000000981:diagnostic imaging; D015588:Observer Variation; D012372:ROC Curve; D013902:Radiography, Thoracic / Q000592:standards*