Degeneracy and discrete receiver operating characteristic rating data.
OBJECTIVE
Observer performance studies sometimes use too few cases for estimating diagnostic accuracy from binormal receiver operating characteristic (ROC) curves. One important problem is degenerate data sets. We compared a new algorithm, RSCORE4, with the exact-solution approach to degeneracy in ROCFIT and with the Wilcoxon statistic.


METHODS
Degenerate ROC solutions result from empty cells in the data matrix. We addressed this problem by adding a small constant to empty cells in a maximum-likelihood program, RSCORE4. When this method failed, the program branched to a pattern-search algorithm. We tested the program in a series of Monte Carlo studies.


RESULTS
RSCORE4 converged to nondegenerate solutions in every case and gave results closer to population values than ROCFIT or Wilcoxon. ROCFIT converged to exact-fit degenerate solutions, those with zero or infinite parameter values, in more than 40% of the samples. The Wilcoxon statistic was biased.


CONCLUSIONS
RSCORE4 seems to outperform other currently recommended methods for dealing with degeneracy.
Dorfman
D D
DD
Department of Psychology, University of Iowa, Iowa City 52242, USA.
Berbaum
K S
KS
D000465:Algorithms; D016013:Likelihood Functions; D009010:Monte Carlo Method; D012372:ROC Curve; D011859:Radiography; D018709:Statistics, Nonparametric