Variable selection under multiple imputation using the bootstrap in a prognostic study.
BACKGROUND
Missing data is a challenging problem in many prognostic studies. Multiple imputation (MI) accounts for imputation uncertainty that allows for adequate statistical testing. We developed and tested a methodology combining MI with bootstrapping techniques for studying prognostic variable selection.


METHODS
In our prospective cohort study we merged data from three different randomized controlled trials (RCTs) to assess prognostic variables for chronicity of low back pain. Among the outcome and prognostic variables data were missing in the range of 0 and 48.1%. We used four methods to investigate the influence of respectively sampling and imputation variation: MI only, bootstrap only, and two methods that combine MI and bootstrapping. Variables were selected based on the inclusion frequency of each prognostic variable, i.e. the proportion of times that the variable appeared in the model. The discriminative and calibrative abilities of prognostic models developed by the four methods were assessed at different inclusion levels.


RESULTS
We found that the effect of imputation variation on the inclusion frequency was larger than the effect of sampling variation. When MI and bootstrapping were combined at the range of 0% (full model) to 90% of variable selection, bootstrap corrected c-index values of 0.70 to 0.71 and slope values of 0.64 to 0.86 were found.


CONCLUSIONS
We recommend to account for both imputation and sampling variation in sets of missing data. The new procedure of combining MI with bootstrapping for variable selection, results in multivariable prognostic models with good performance and is therefore attractive to apply on data sets with missing values.
Heymans
Martijn W
MW
Vrije Universiteit, Institute for Health Sciences, Department of Methodology and Applied Biostatistics, Amsterdam, The Netherlands. mw.heymans@vumc.nl
van Buuren
Stef
S
Knol
Dirk L
DL
van Mechelen
Willem
W
de Vet
Henrica C W
HC
D001699:Biometry / Q000379:methods*; D002908:Chronic Disease; D015331:Cohort Studies; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D017116:Low Back Pain / Q000175:diagnosis* / Q000628:therapy; D008297:Male; D015233:Models, Statistical; D009426:Netherlands; D017063:Outcome Assessment, Health Care / Q000379:methods* / Q000706:statistics & numerical data; D011379:Prognosis; D011446:Prospective Studies; D016032:Randomized Controlled Trials as Topic; D035501:Uncertainty