An equivalent relative utility metric for evaluating screening mammography.
Comparative studies of performance in screening mammography are often ambiguous. A new method will frequently show a higher sensitivity or detection rate than an existing standard with a concomitant increase in false positives or recalls. The authors propose an equivalent relative utility (ERU) metric based on signal detection theory to quantify screening performance in such comparisons. The metric is defined as the relative utility, as defined in classical signal detection theory, needed to make 2 systems equivalent. ERU avoids the problem of requiring a predefined putative relative utility, which has limited application of utility theory in receiver operating characteristic analysis. The metric can be readily estimated from recall and detection rates commonly reported in comparative clinical studies. An important practical advantage of ERU is that in prevalence matched populations, the measure can be estimated without an independent estimate of disease prevalence. Thus estimating ERU does not require a study with long-term follow-up to find cases of missed disease. The approach is applicable to any comparative screening study that reports results in terms of recall and detection rates, although the authors focus exclusively on screening mammography in this work. They derive the ERU from the definition of utility given in classical treatments of signal detection theory. They also investigate reasonable values of relative utility in screening mammography for use in interpreting ERU using data from a large clinical study. As examples of application of ERU, they reanalyze 2 recently published reports using recall and detection rates in screening mammography.
Abbey
Craig K
CK
Department of Biomedical Engineering, University of California, Davis, CA, USA. abbey@psych.ucsb.edu
Eckstein
Miguel P
MP
Boone
John M
JM
D000465:Algorithms; D001943:Breast Neoplasms / Q000175:diagnosis*; D003662:Decision Theory; D005260:Female; D006801:Humans; D008327:Mammography / Q000592:standards*; D008403:Mass Screening / Q000592:standards*; D012372:ROC Curve; D012189:Retrospective Studies; D017603:Signal Detection, Psychological
Cancer. 2004 Apr 15;100(8):1590-4
15073844
Semin Nucl Med. 1978 Oct;8(4):283-98
112681
J Natl Cancer Inst. 2004 Dec 15;96(24):1840-50
15601640
Int J Biomed Comput. 1989 Sep;24(3):153-89
2509379
Radiol Phys Technol. 2008 Jan;1(1):2-12
20821157
N Engl J Med. 2003 Apr 24;348(17):1672-80
12711743
Acad Radiol. 1996 Mar;3(3):245-53
8796672
Ann Intern Med. 2002 Sep 3;137(5 Part 1):347-60
12204020
Stat Med. 1998 May 15;17(9):1033-53
9612889
Med Decis Making. 2004 Nov-Dec;24(6):561-72
15534338
N Engl J Med. 2005 Oct 27;353(17):1784-92
16251534