title: An equivalent relative utility metric for evaluating screening mammography.
abstract: Comparative studies of performance in screening mammography are often ambiguous. A new method will frequently show a higher sensitivity or detection rate than an existing standard with a concomitant increase in false positives or recalls. The authors propose an equivalent relative utility (ERU) metric based on signal detection theory to quantify screening performance in such comparisons. The metric is defined as the relative utility, as defined in classical signal detection theory, needed to make 2 systems equivalent. ERU avoids the problem of requiring a predefined putative relative utility, which has limited application of utility theory in receiver operating characteristic analysis. The metric can be readily estimated from recall and detection rates commonly reported in comparative clinical studies. An important practical advantage of ERU is that in prevalence matched populations, the measure can be estimated without an independent estimate of disease prevalence. Thus estimating ERU does not require a study with long-term follow-up to find cases of missed disease. The approach is applicable to any comparative screening study that reports results in terms of recall and detection rates, although the authors focus exclusively on screening mammography in this work. They derive the ERU from the definition of utility given in classical treatments of signal detection theory. They also investigate reasonable values of relative utility in screening mammography for use in interpreting ERU using data from a large clinical study. As examples of application of ERU, they reanalyze 2 recently published reports using recall and detection rates in screening mammography.
authors.affiliation: Department of Biomedical Engineering, University of California, Davis, CA, USA. abbey@psych.ucsb.edu
authors.forename: Craig K
authors.initials: CK
authors.lastname: Abbey
authors.forename: Miguel P
authors.initials: MP
authors.lastname: Eckstein
authors.forename: John M
authors.initials: JM
authors.lastname: Boone
mesh_terms: D000465:Algorithms; D001943:Breast Neoplasms / Q000175:diagnosis*; D003662:Decision Theory; D005260:Female; D006801:Humans; D008327:Mammography / Q000592:standards*; D008403:Mass Screening / Q000592:standards*; D012372:ROC Curve; D012189:Retrospective Studies; D017603:Signal Detection, Psychological
references.citation: Cancer. 2004 Apr 15;100(8):1590-4
references.pmid: 15073844
references.citation: Semin Nucl Med. 1978 Oct;8(4):283-98
references.pmid: 112681
references.citation: J Natl Cancer Inst. 2004 Dec 15;96(24):1840-50
references.pmid: 15601640
references.citation: Int J Biomed Comput. 1989 Sep;24(3):153-89
references.pmid: 2509379
references.citation: Radiol Phys Technol. 2008 Jan;1(1):2-12
references.pmid: 20821157
references.citation: N Engl J Med. 2003 Apr 24;348(17):1672-80
references.pmid: 12711743
references.citation: Acad Radiol. 1996 Mar;3(3):245-53
references.pmid: 8796672
references.citation: Ann Intern Med. 2002 Sep 3;137(5 Part 1):347-60
references.pmid: 12204020
references.citation: Stat Med. 1998 May 15;17(9):1033-53
references.pmid: 9612889
references.citation: Med Decis Making. 2004 Nov-Dec;24(6):561-72
references.pmid: 15534338
references.citation: N Engl J Med. 2005 Oct 27;353(17):1784-92
references.pmid: 16251534