title: Is the area under an ROC curve a valid measure of the performance of a screening or diagnostic test?
abstract: OBJECTIVE
  The area under a receiver operating characteristic (ROC) curve (the AUC) is used as a measure of the performance of a screening or diagnostic test. We here assess the validity of the AUC.
  
  
  METHODS
  Assuming the test results follow Gaussian distributions in affected and unaffected individuals, standard mathematical formulae were used to describe the relationship between the detection rate (DR) (or sensitivity) and the false-positive rate (FPR) of a test with the AUC. These formulae were used to calculate the screening performance (DR for a given FPR, or FPR for a given DR) for different AUC values according to different standard deviations of the test result in affected and unaffected individuals.
  
  
  RESULTS
  The DR for a given FPR is strongly dependent on relative differences in the standard deviation of the test variable in affected and unaffected individuals. Consequently, two tests with the same AUC can have a different DR for the same FPR. For example, an AUC of 0.75 has a DR of 24% for a 5% FPR if the standard deviations are the same in affected and unaffected individuals, but 39% for the same 5% FPR if the standard deviation in affected individuals is 1.5 times that in unaffected individuals.
  
  
  CONCLUSIONS
  The AUC is an unreliable measure of screening performance because in practice the standard deviation of a screening or diagnostic test in affected and unaffected individuals can differ. The problem is avoided by not using AUC at all, and instead specifying DRs for given FPRs or FPRs for given DRs.
authors.affiliation: Wolfson Institute of Preventive Medicine, Barts and the London School of Medicine and Dentistry, Charterhouse Square, EC1M 6BQ, London.
authors.forename: N J
authors.initials: NJ
authors.lastname: Wald
authors.forename: J P
authors.initials: JP
authors.lastname: Bestwick
mesh_terms: D019540:Area Under Curve; D003955:Diagnostic Tests, Routine / Q000379:methods* / Q000592:standards; D005189:False Positive Reactions; D006801:Humans; D008403:Mass Screening / Q000379:methods*; D008962:Models, Theoretical; D016011:Normal Distribution; D012372:ROC Curve; D015203:Reproducibility of Results
keywords: AUC; ROC curve; diagnostic test; screening test