title: Statistical power considerations for a utility endpoint in observer performance studies.
abstract: OBJECTIVE
  The purpose of this investigation is to compare the statistical power of the most common measure of performance for observer performance studies, area under the ROC curve (AUC), to an expected utility (EU) endpoint.
  
  
  METHODS
  We have modified a well-known simulation procedure developed by Roe and Metz for statistical power analysis in receiver operating characteristic (ROC) studies. Starting from a set of baseline simulations, we investigate the effects of three parameters that describe properties of the observers (iso-utility slope, unequal variance, and tendency to favor more aggressive or conservative actions) and three parameters that affect experimental design (number of readers, number of cases, and fraction of positive cases).
  
  
  RESULTS
  The EU endpoint generally has good statistical power relative to AUC in our simulations. Of 396 total conditions simulated, EU had higher statistical power in 377 cases (95%). In 246 of these cases, EU power was 5 percentage points or more higher than AUC. In simulation runs evaluating the effect of the number of readers and cases on the baseline simulations, EU measure had equivalent power to AUC with fewer readers (9% to 28%) or fewer cases (18% to 41%).
  
  
  CONCLUSIONS
  These simulation studies provide further motivation for considering EU in studies of screening mammography technology and they motivate investigations of utility in other diagnostic tasks.
authors.affiliation: Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA, USA. abbey@psych.ucsb.edu
authors.forename: Craig K
authors.initials: CK
authors.lastname: Abbey
authors.forename: Frank W
authors.initials: FW
authors.lastname: Samuelson
authors.forename: Brandon D
authors.initials: BD
authors.lastname: Gallas
mesh_terms: D003198:Computer Simulation; D015233:Models, Statistical; D012372:ROC Curve; D011871:Radiology / Q000379:methods* / Q000706:statistics & numerical data*; D013223:Statistics as Topic / Q000379:methods*