Subsampling versus bootstrapping in resampling-based model selection for multivariable regression.
In recent years, increasing attention has been devoted to the problem of the stability of multivariable regression models, understood as the resistance of the model to small changes in the data on which it has been fitted. Resampling techniques, mainly based on the bootstrap, have been developed to address this issue. In particular, the approaches based on the idea of "inclusion frequency" consider the repeated implementation of a variable selection procedure, for example backward elimination, on several bootstrap samples. The analysis of the variables selected in each iteration provides useful information on the model stability and on the variables' importance. Recent findings, nevertheless, show possible pitfalls in the use of the bootstrap, and alternatives such as subsampling have begun to be taken into consideration in the literature. Using model selection frequencies and variable inclusion frequencies, we empirically compare these two different resampling techniques, investigating the effect of their use in selected classical model selection procedures for multivariable regression. We conduct our investigations by analyzing two real data examples and by performing a simulation study. Our results reveal some advantages in using a subsampling technique rather than the bootstrap in this context.
Department of Medical Informatics, Biometry and Epidemiology, University of Munich, Marchioninistr. 15, 81377 Munich, Germany.
Riccardo
R
De Bin
Department of Medical Informatics, Biometry and Epidemiology, University of Munich, Marchioninistr. 15, 81377 Munich, Germany.
Silke
S
Janitza
Department of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Stefan-Meier-Str. 26, 79106 Freiburg im Breisgau, Germany.
Willi
W
Sauerbrei
Department of Medical Informatics, Biometry and Epidemiology, University of Munich, Marchioninistr. 15, 81377 Munich, Germany.
Anne-Laure
AL
Boulesteix
D000465:Algorithms; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D015233:Models, Statistical; D015999:Multivariate Analysis; D012044:Regression Analysis; D015203:Reproducibility of Results; D018401:Sample Size; D012680:Sensitivity and Specificity
Bootstrap; Model selection; Model stability; Subsampling