A contaminated binormal model for ROC data: Part I. Some interesting examples of binormal degeneracy.
OBJECTIVE
Receiver operating characteristic (ROC) data with false-positive fractions of 0 are often difficult to fit with standard ROC methods and are sometimes discarded. Some extreme examples of such data were analyzed to evaluate the nature of these difficulties.


METHODS
Rating reports of fracture for single-view ankle radiographs were analyzed with the binormal ROC model and with two ROC models that keep the ROC curve from crossing the chance line. Because fractures were almost never reported that were not present, some views and locations yielded only ROC points with false-positive fractions of 0, while others yielded at least one ROC point with a non-0 false-positive fraction.


RESULTS
The models tended to yield ROC areas close to or equal to 1. ROC areas of 1 imply a true-positive fraction close to 1; yet the data contained no such fractions. When all false-positive fractions were 0, the true-positive fraction could be much higher for one view than another for all observers. ROC areas gave little or no hint of these unmistakable differences in performance.


CONCLUSIONS
These data challenge the validity and robustness of current ROC models. A key aspect of ankle fractures is that some may be visible on one view but not at all visible on another.
Dorfman
D D
DD
Department of Radiology, University of Iowa, Iowa City 52242, USA.
Berbaum
K S
KS
Brandser
E A
EA
D016512:Ankle Injuries / Q000000981:diagnostic imaging*; D001175:Arthrography; D003662:Decision Theory; D003937:Diagnosis, Differential; D050723:Fractures, Bone / Q000000981:diagnostic imaging*; D006801:Humans; D015588:Observer Variation; D012372:ROC Curve