title: A scaling transformation for classifier output based on likelihood ratio: applications to a CAD workstation for diagnosis of breast cancer.
abstract: OBJECTIVE
  The authors developed scaling methods that monotonically transform the output of one classifier to the "scale" of another. Such transformations affect the distribution of classifier output while leaving the ROC curve unchanged. In particular, they investigated transformations between radiologists and computer classifiers, with the goal of addressing the problem of comparing and interpreting case-specific values of output from two classifiers.
  
  
  METHODS
  Using both simulated and radiologists' rating data of breast imaging cases, the authors investigated a likelihood-ratio-scaling transformation, based on "matching" classifier likelihood ratios. For comparison, three other scaling transformations were investigated that were based on matching classifier true positive fraction, false positive fraction, or cumulative distribution function, respectively. The authors explored modifying the computer output to reflect the scale of the radiologist, as well as modifying the radiologist's ratings to reflect the scale of the computer. They also evaluated how dataset size affects the transformations.
  
  
  RESULTS
  When ROC curves of two classifiers differed substantially, the four transformations were found to be quite different. The likelihood-ratio scaling transformation was found to vary widely from radiologist to radiologist. Similar results were found for the other transformations. Our simulations explored the effect of database sizes on the accuracy of the estimation of our scaling transformations.
  
  
  CONCLUSIONS
  The likelihood-ratio-scaling transformation that the authors have developed and evaluated was shown to be capable of transforming computer and radiologist outputs to a common scale reliably, thereby allowing the comparison of the computer and radiologist outputs on the basis of a clinically relevant statistic.
authors.affiliation: Department of Radiology, The University of Chicago, Chicago, IL 60637, USA.
authors.forename: Karla
authors.initials: K
authors.lastname: Horsch
authors.forename: Lorenzo L
authors.initials: LL
authors.lastname: Pesce
authors.forename: Maryellen L
authors.initials: ML
authors.lastname: Giger
authors.forename: Charles E
authors.initials: CE
authors.lastname: Metz
authors.forename: Yulei
authors.initials: Y
authors.lastname: Jiang
mesh_terms: D019540:Area Under Curve; D001499:Bayes Theorem; D001943:Breast Neoplasms / Q000175:diagnosis*; D003196:Computer Graphics; D003936:Diagnosis, Computer-Assisted / Q000379:methods*; D006801:Humans; D016013:Likelihood Functions; D012189:Retrospective Studies; D012680:Sensitivity and Specificity