title: Prediction impact curve is a new measure integrating intervention effects in the evaluation of risk models.
abstract: OBJECTIVE
  We propose a new measure of assessing the performance of risk models, the area under the prediction impact curve (auPIC), which quantifies the performance of risk models in terms of their average health impact in the population.
  
  
  METHODS
  Using simulated data, we explain how the prediction impact curve (PIC) estimates the percentage of events prevented when a risk model is used to assign high-risk individuals to an intervention. We apply the PIC to the Atherosclerosis Risk in Communities (ARIC) Study to illustrate its application toward prevention of coronary heart disease.
  
  
  RESULTS
  We estimated that if the ARIC cohort received statins at baseline, 5% of events would be prevented when the risk model was evaluated at a cutoff threshold of 20% predicted risk compared to 1% when individuals were assigned to the intervention without the use of a model. By calculating the auPIC, we estimated that an average of 15% of events would be prevented when considering performance across the entire interval.
  
  
  CONCLUSIONS
  We conclude that the PIC is a clinically meaningful measure for quantifying the expected health impact of risk models that supplements existing measures of model performance.
authors.affiliation: Department of Epidemiology, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA, USA.
authors.forename: William
authors.initials: W
authors.lastname: Campbell
authors.affiliation: Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, PO Box 281, SE-171 77 Stockholm, Sweden; Department of Medical Sciences, Molecular Epidemiology and Science for Life Laboratory, Uppsala University, BOX 1115, 751 41 Uppsala, Sweden.
authors.forename: Andrea
authors.initials: A
authors.lastname: Ganna
authors.affiliation: Department of Medical Sciences, Molecular Epidemiology and Science for Life Laboratory, Uppsala University, BOX 1115, 751 41 Uppsala, Sweden.
authors.forename: Erik
authors.initials: E
authors.lastname: Ingelsson
authors.affiliation: Department of Epidemiology, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA, USA; Department of Clinical Genetics, Section Community Genetics, EMGO Institute for Health and Care Research, VU University Medical Center, PO Box 7057, 1007 MB Amsterdam, The Netherlands. Electronic address: cecile.janssens@emory.edu.
authors.forename: A Cecile J W
authors.initials: AC
authors.lastname: Janssens
mesh_terms: D019540:Area Under Curve; D050197:Atherosclerosis; D003327:Coronary Disease; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D011379:Prognosis; D018570:Risk Assessment
keywords: AUC; Coronary heart disease; Prediction impact curve; Predictive ability; Predictive model; Risk model