title: "Binary" and "non-binary" detection tasks: are current performance measures optimal?
abstract: We have observed that a very large fraction of responses for several detection tasks during the performance of observer studies are in the extreme ranges of lower than 11% or higher than 89% regardless of the actual presence or absence of the abnormality in question or its subjectively rated "subtleness." This observation raises questions regarding the validity and appropriateness of using multicategory rating scales for such detection tasks. Monte Carlo simulation of binary and multicategory ratings for these tasks demonstrate that the use of the former (binary) often results in a less biased and more precise summary index and hence may lead to a higher statistical power for determining differences between modalities.
authors.affiliation: Department of Radiology, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA. gurd@upmc.edu
authors.forename: David
authors.initials: D
authors.lastname: Gur
authors.forename: Howard E
authors.initials: HE
authors.lastname: Rockette
authors.forename: Andriy I
authors.initials: AI
authors.lastname: Bandos
mesh_terms: D019540:Area Under Curve; D002983:Clinical Competence; D003936:Diagnosis, Computer-Assisted; D003952:Diagnostic Imaging; D006801:Humans; D008171:Lung Diseases / Q000175:diagnosis*; D009010:Monte Carlo Method; D015588:Observer Variation; D012372:ROC Curve; D015203:Reproducibility of Results