title: The residual-based predictiveness curve: A visual tool to assess the performance of prediction models.
abstract: It is agreed among biostatisticians that prediction models for binary outcomes should satisfy two essential criteria: first, a prediction model should have a high discriminatory power, implying that it is able to clearly separate cases from controls. Second, the model should be well calibrated, meaning that the predicted risks should closely agree with the relative frequencies observed in the data. The focus of this work is on the predictiveness curve, which has been proposed by Huang et al. (Biometrics 63, 2007) as a graphical tool to assess the aforementioned criteria. By conducting a detailed analysis of its properties, we review the role of the predictiveness curve in the performance assessment of biomedical prediction models. In particular, we demonstrate that marker comparisons should not be based solely on the predictiveness curve, as it is not possible to consistently visualize the added predictive value of a new marker by comparing the predictiveness curves obtained from competing models. Based on our analysis, we propose the "residual-based predictiveness curve" (RBP curve), which addresses the aforementioned issue and which extends the original method to settings where the evaluation of a prediction model on independent test data is of particular interest. Similar to the predictiveness curve, the RBP curve reflects both the calibration and the discriminatory power of a prediction model. In addition, the curve can be conveniently used to conduct valid performance checks and marker comparisons.
authors.affiliation: Department of Statistics, University of Munich, 80539 Munich, Germany.
authors.forename: Giuseppe
authors.initials: G
authors.lastname: Casalicchio
authors.affiliation: Department of Statistics, University of Munich, 80539 Munich, Germany.
authors.forename: Bernd
authors.initials: B
authors.lastname: Bischl
authors.affiliation: Department of Medical Informatics, Biometry and Epidemiology, University of Munich, 81377 Munich, Germany.
authors.forename: Anne-Laure
authors.initials: AL
authors.lastname: Boulesteix
authors.affiliation: Department of Medical Biometry, Informatics and Epidemiology, University of Bonn, 53105 Bonn, Germany.
authors.forename: Matthias
authors.initials: M
authors.lastname: Schmid
mesh_terms: D015415:Biomarkers / Q000032:analysis*; D001699:Biometry / Q000379:methods; D003627:Data Interpretation, Statistical; D003936:Diagnosis, Computer-Assisted / Q000379:methods; D006801:Humans; D015233:Models, Statistical; D011379:Prognosis; D012680:Sensitivity and Specificity
chemical_list: D015415:Biomarkers
keywords: Calibration; Classification; Discrimination; Predictiveness curve; Risk prediction