title: Doubly Optimized Calibrated Support Vector Machine (DOC-SVM): an algorithm for joint optimization of discrimination and calibration.
abstract: Historically, probabilistic models for decision support have focused on discrimination, e.g., minimizing the ranking error of predicted outcomes. Unfortunately, these models ignore another important aspect, calibration, which indicates the magnitude of correctness of model predictions. Using discrimination and calibration simultaneously can be helpful for many clinical decisions. We investigated tradeoffs between these goals, and developed a unified maximum-margin method to handle them jointly. Our approach called, Doubly Optimized Calibrated Support Vector Machine (DOC-SVM), concurrently optimizes two loss functions: the ridge regression loss and the hinge loss. Experiments using three breast cancer gene-expression datasets (i.e., GSE2034, GSE2990, and Chanrion's datasets) showed that our model generated more calibrated outputs when compared to other state-of-the-art models like Support Vector Machine (p=0.03, p=0.13, and p<0.001) and Logistic Regression (p=0.006, p=0.008, and p<0.001). DOC-SVM also demonstrated better discrimination (i.e., higher AUCs) when compared to Support Vector Machine (p=0.38, p=0.29, and p=0.047) and Logistic Regression (p=0.38, p=0.04, and p<0.0001). DOC-SVM produced a model that was better calibrated without sacrificing discrimination, and hence may be helpful in clinical decision making.
authors.affiliation: Division of Biomedical Informatics, University California San Diego (UCSD), La Jolla, California, USA. x1jiang@ucsd.edu
authors.forename: Xiaoqian
authors.initials: X
authors.lastname: Jiang
authors.forename: Aditya
authors.initials: A
authors.lastname: Menon
authors.forename: Shuang
authors.initials: S
authors.lastname: Wang
authors.forename: Jihoon
authors.initials: J
authors.lastname: Kim
authors.forename: Lucila
authors.initials: L
authors.lastname: Ohno-Machado
mesh_terms: D019540:Area Under Curve; D001943:Breast Neoplasms / Q000235:genetics; D002138:Calibration; D003198:Computer Simulation; D030541:Databases, Genetic; D005260:Female; D006801:Humans; D008954:Models, Biological; D011336:Probability; D012372:ROC Curve; D015203:Reproducibility of Results; D060388:Support Vector Machine