title: Calibrating predictive model estimates to support personalized medicine.
abstract: OBJECTIVE
  Predictive models that generate individualized estimates for medically relevant outcomes are playing increasing roles in clinical care and translational research. However, current methods for calibrating these estimates lose valuable information. Our goal is to develop a new calibration method to conserve as much information as possible, and would compare favorably to existing methods in terms of important performance measures: discrimination and calibration.
  
  
  METHODS
  We propose an adaptive technique that utilizes individualized confidence intervals (CIs) to calibrate predictions. We evaluate this new method, adaptive calibration of predictions (ACP), in artificial and real-world medical classification problems, in terms of areas under the ROC curves, the Hosmer-Lemeshow goodness-of-fit test, mean squared error, and computational complexity.
  
  
  RESULTS
  ACP compared favorably to other calibration methods such as binning, Platt scaling, and isotonic regression. In several experiments, binning, isotonic regression, and Platt scaling failed to improve the calibration of a logistic regression model, whereas ACP consistently improved the calibration while maintaining the same discrimination or even improving it in some experiments. In addition, the ACP algorithm is not computationally expensive.
  
  
  CONCLUSIONS
  The calculation of CIs for individual predictions may be cumbersome for certain predictive models. ACP is not completely parameter-free: the length of the CI employed may affect its results.
  
  
  CONCLUSIONS
  ACP can generate estimates that may be more suitable for individualized predictions than estimates that are calibrated using existing methods. Further studies are necessary to explore the limitations of ACP.
authors: Division of Biomedical Informatics, School of Medicine, University of California, San Diego, La Jolla, California 92093, USA. x1jiang@ucsd.edu
authors: Xiaoqian
authors: X
authors: Jiang
authors: Melanie
authors: M
authors: Osl
authors: Jihoon
authors: J
authors: Kim
authors: Lucila
authors: L
authors: Ohno-Machado
mesh_terms: D016001:Confidence Intervals; D006801:Humans; D016015:Logistic Models; D055641:Mathematical Concepts; D008962:Models, Theoretical; D010351:Patient Discharge / Q000706:statistics & numerical data; D057285:Precision Medicine