title: Statistical variation in progressive scrambling.
abstract: The two methods most often used to evaluate the robustness and predictivity of partial least squares (PLS) models are cross-validation and response randomization. Both methods may be overly optimistic for data sets that contain redundant observations, however. The kinds of perturbation analysis widely used for evaluating model stability in the context of ordinary least squares regression are only applicable when the descriptors are independent of each other and errors are independent and normally distributed; neither assumption holds for QSAR in general and for PLS in particular. Progressive scrambling is a novel, nonparametric approach to perturbing models in the response space in a way that does not disturb the underlying covariance structure of the data. Here, we introduce adjustments for two of the characteristic values produced by a progressive scrambling analysis - the deprecated predictivity (Q*2s) and standard error of prediction (SDEPs*) - that correct for the effect of introduced perturbation. We also explore the statistical behavior of the adjusted values (Q*2(0) and SDEP0*) and the sensitivity to perturbation (dq2/dryy'2). It is shown that the three statistics are all robust for stable PLS models, in terms of the stochastic component of their determination and of their variation due to sampling effects involved in training set selection.
authors.affiliation: Tripos, Inc., 1699 S. Hanley Road, St. Louis, MO 63144, USA. bclark@tripos.com
authors.forename: Robert D
authors.initials: RD
authors.lastname: Clark
authors.forename: Peter C
authors.initials: PC
authors.lastname: Fox
mesh_terms: D003627:Data Interpretation, Statistical; D016018:Least-Squares Analysis; D021281:Quantitative Structure-Activity Relationship
references.citation: J Chem Inf Comput Sci. 2004 Jan-Feb;44(1):1-12
references.pmid: 14741005
references.citation: J Med Chem. 1997 Dec 19;40(26):4257-64
references.pmid: 9435895
references.citation: J Comput Aided Mol Des. 2003 Feb-Apr;17(2-4):265-75
references.pmid: 13677492
references.citation: J Chem Inf Comput Sci. 2003 Mar-Apr;43(2):579-86
references.pmid: 12653524
references.citation: J Am Chem Soc. 1988 Aug 1;110(18):5959-67
references.pmid: 22148765
references.citation: J Med Chem. 2001 Sep 27;44(20):3223-30
references.pmid: 11563921
references.citation: J Comput Aided Mol Des. 1993 Oct;7(5):587-619
references.pmid: 8294948
references.citation: J Comput Aided Mol Des. 2004 Jul-Sep;18(7-9):537-47
references.pmid: 15729853
references.citation: J Mol Graph Model. 2002 Jan;20(4):269-76
references.pmid: 11858635
references.citation: J Chem Inf Comput Sci. 1997 Mar-Apr;37(2):392-401
references.pmid: 9090857