title: The importance of prediction model validation and assessment in obesity and nutrition research.
abstract: Deriving statistical models to predict one variable from one or more other variables, or predictive modeling, is an important activity in obesity and nutrition research. To determine the quality of the model, it is necessary to quantify and report the predictive validity of the derived models. Conducting validation of the predictive measures provides essential information to the research community about the model. Unfortunately, many articles fail to account for the nearly inevitable reduction in predictive ability that occurs when a model derived on one data set is applied to a new data set. Under some circumstances, the predictive validity can be reduced to nearly zero. In this overview, we explain why reductions in predictive validity occur, define the metrics commonly used to estimate the predictive validity of a model (for example, coefficient of determination (R(2)), mean squared error, sensitivity, specificity, receiver operating characteristic and concordance index) and describe methods to estimate the predictive validity (for example, cross-validation, bootstrap, and adjusted and shrunken R(2)). We emphasize that methods for estimating the expected reduction in predictive ability of a model in new samples are available and this expected reduction should always be reported when new predictive models are introduced.
authors.affiliation: Department of Mathematical Sciences, Montclair State University, Montclair, NJ, USA.
authors.forename: A E
authors.initials: AE
authors.lastname: Ivanescu
authors.affiliation: Office of Energetics and Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA.
authors.forename: P
authors.identifier: 0000-0002-9026-9999
authors.initials: P
authors.lastname: Li
authors.affiliation: Office of Energetics and Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA.
authors.forename: B
authors.initials: B
authors.lastname: George
authors.affiliation: Office of Energetics and Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA.
authors.forename: A W
authors.initials: AW
authors.lastname: Brown
authors.affiliation: Division of Biostatistics, Department of Pharmacology and Experimental Therapeutics, Thomas Jefferson University, Philadelphia, PA, USA.
authors.forename: S W
authors.initials: SW
authors.lastname: Keith
authors.affiliation: School of Nursing, University of Alabama at Birmingham, Birmingham, AL, USA.
authors.forename: D
authors.initials: D
authors.lastname: Raju
authors.affiliation: Office of Energetics and Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA.|Department of Biostatistics, University of Alabama at Birmingham, Birmingham, AL, USA.
authors.forename: D B
authors.initials: DB
authors.lastname: Allison
mesh_terms: D035843:Biomedical Research / Q000379:methods* / Q000592:standards*; D006801:Humans; D015233:Models, Statistical; D052756:Nutritional Sciences / Q000592:standards*; D009765:Obesity; D011237:Predictive Value of Tests; D015203:Reproducibility of Results