title: Prediction of driving ability: Are we building valid models?
abstract: The prediction of on-road driving ability using off-road measures is a key aim in driving research. The primary goal in most classification models is to determine a small number of off-road variables that predict driving ability with high accuracy. Unfortunately, classification models are often over-fitted to the study sample, leading to inflation of predictive accuracy, poor generalization to the relevant population and, thus, poor validity. Many driving studies do not report sufficient details to determine the risk of model over-fitting and few report any validation technique, which is critical to test the generalizability of a model. After reviewing the literature, we generated a model using a moderately large sample size (n=279) employing best practice techniques in the context of regression modelling. By then randomly selecting progressively smaller sample sizes we show that a low ratio of participants to independent variables can result in over-fitted models and spurious conclusions regarding model accuracy. We conclude that more stable models can be constructed by following a few guidelines.
authors: New Zealand Brain Research Institute, Christchurch, New Zealand; Psychiatric Service for the Elderly, The Princess Margaret Hospital, Christchurch, New Zealand. Electronic address: petra.hoggarth@cdhb.govt.nz.
authors: Petra A
authors: PA
authors: Hoggarth
authors: New Zealand Brain Research Institute, Christchurch, New Zealand; Department of Medical Physics and Bioengineering, Christchurch Hospital, Christchurch, New Zealand; Department of Electrical and Computer Engineering, University of Canterbury, Christchurch, New Zealand.
authors: Carrie R H
authors: CR
authors: Innes
authors: New Zealand Brain Research Institute, Christchurch, New Zealand; Department of Psychology, University of Canterbury, Christchurch, New Zealand; Department of Medicine, University of Otago, Christchurch, New Zealand.
authors: John C
authors: JC
authors: Dalrymple-Alford
authors: New Zealand Brain Research Institute, Christchurch, New Zealand; Department of Medical Physics and Bioengineering, Christchurch Hospital, Christchurch, New Zealand; Department of Electrical and Computer Engineering, University of Canterbury, Christchurch, New Zealand; Department of Psychology, University of Canterbury, Christchurch, New Zealand; Department of Medicine, University of Otago, Christchurch, New Zealand.
authors: Richard D
authors: RD
authors: Jones
mesh_terms: D000368:Aged; D000369:Aged, 80 and over; D001334:Automobile Driving; D005260:Female; D006801:Humans; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D008960:Models, Psychological; D009520:New Zealand; D011597:Psychomotor Performance / Q000502:physiology*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
keywords: Driving; Modeling; Over-fitting; Prediction; Regression