Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis.
OBJECTIVE
The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments.


METHODS
We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method.


RESULTS
We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure.


CONCLUSIONS
We recommend using the modified DBM procedure that incorporates the recent developments.
Hillis
Stephen L
SL
Center for Research in the Implementation of Innovative Strategies in Practice (CRIISP), Iowa City VA Medical Center (152), Iowa City, IA, USA. steve-hillis@uiowa.edu
Berbaum
Kevin S
KS
Metz
Charles E
CE
D000328:Adult; D000465:Algorithms; D000704:Analysis of Variance; D000784:Aortic Dissection / Q000175:diagnosis; D001014:Aortic Aneurysm / Q000175:diagnosis; D003198:Computer Simulation; D006801:Humans; D007231:Infant, Newborn; D007232:Infant, Newborn, Diseases / Q000175:diagnosis; D012372:ROC Curve; D011859:Radiography / Q000706:statistics & numerical data*; D011873:Radiology Information Systems