Free-response methodology: alternate analysis and a new observer-performance experiment.
The authors review the recently introduced method for analyzing free-response receiver operating characteristic (FROC) observer-performance experiments. These experiments allow multiple abnormalities and observer responses per image and require correct localization for true-positive events. A fundamental plot, the alternative FROC (AFROC), is emphasized, the area under which is a natural index of performance for measuring FROC observer performance. The authors show that it is possible to analyze free-response data in more general ways. The first method, AFROC analysis, involves scoring FROC images in a different way. The second method involves a new observer-performance experiment, the free-response forced error (FFE) experiment. FROC and AFROC analysis was applied to several FROC data sets, and simultaneous FROC-AFROC-FFE experiments were performed with several radiologist observers to test the three methods. The methods were found to be in good agreement with one another.
Chakraborty
D P
DP
Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia 19104.
Winter
L H
LH
D006801:Humans; D015588:Observer Variation; D012372:ROC Curve; D011859:Radiography; D012984:Software