title: Assessing model fit by cross-validation.
abstract: When QSAR models are fitted, it is important to validate any fitted model-to check that it is plausible that its predictions will carry over to fresh data not used in the model fitting exercise. There are two standard ways of doing this-using a separate hold-out test sample and the computationally much more burdensome leave-one-out cross-validation in which the entire pool of available compounds is used both to fit the model and to assess its validity. We show by theoretical argument and empiric study of a large QSAR data set that when the available sample size is small-in the dozens or scores rather than the hundreds, holding a portion of it back for testing is wasteful, and that it is much better to use cross-validation, but ensure that this is done properly.
authors.affiliation: School of Statistics, University of Minnesota, Minneapolis, Minnesota 55455, USA. doug@stat.umn.edu
authors.forename: Douglas M
authors.initials: DM
authors.lastname: Hawkins
authors.forename: Subhash C
authors.initials: SC
authors.lastname: Basak
authors.forename: Denise
authors.initials: D
authors.lastname: Mills