Reducing variability in the output of pattern classifiers using histogram shaping.
OBJECTIVE
The authors present a novel technique based on histogram shaping to reduce the variability in the output and (sensitivity, specificity) pairs of pattern classifiers with identical ROC curves, but differently distributed outputs.


METHODS
The authors identify different sources of variability in the output of linear pattern classifiers with identical ROC curves, which also result in classifiers with differently distributed outputs. They theoretically develop a novel technique based on the matching of the histograms of these differently distributed pattern classifier outputs to reduce the variability in their (sensitivity, specificity) pairs at fixed decision thresholds, and to reduce the variability in their actual output values. They empirically demonstrate the efficacy of the proposed technique by means of analyses on the simulated data and real world mammography data.


RESULTS
For the simulated data, with three different known sources of variability, and for the real world mammography data with unknown sources of variability, the proposed classifier output calibration technique significantly reduced the variability in the classifiers' (sensitivity, specificity) pairs at fixed decision thresholds. Furthermore, for classifiers with monotonically or approximately monotonically related output variables, the histogram shaping technique also significantly reduced the variability in their actual output values.


CONCLUSIONS
Classifier output calibration based on histogram shaping can be successfully employed to reduce the variability in the output values and (sensitivity, specificity) pairs of pattern classifiers with identical ROC curves, but differently distributed outputs.
Gupta
Shalini
S
Wireless Terminals Business Unit, Texas Instruments Incorporated, 12500 TI Boulevard, Dallas, Texas 75243, USA. shalini.gupta@ieee.org
Kan
Chih-Wen
CW
Markey
Mia K
MK
D000465:Algorithms; D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000175:diagnosis / Q000473:pathology*; D002138:Calibration; D003198:Computer Simulation; D005260:Female; D006801:Humans; D008327:Mammography / Q000379:methods*; D015233:Models, Statistical; D016011:Normal Distribution; D010363:Pattern Recognition, Automated; D011336:Probability; D012372:ROC Curve; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity