Logistic regression model: an assessment of variability of predictions.
Risk prediction models available for cardiovascular prevention are statistical or based on machine learning methods. This paper investigates whether the logistic regression method can be considered as reference for validation of other methods. In order to test the stability of the predictions using this method, we performed two types of analyses on 50 random training and test samples drawn from the same database. In first analyses three models were obtained by forced entry of different sets of four variables. In second analyses, models were built with increasing number of predictive variables. The predictive performance was assessed by the area under the ROC curve. Although across-samples variability is low for a given model, it is large enough to lead to wrong conclusions when comparing different prediction methods. We also suggest that a low events-per-variable ratio alters the stability of a model's coefficients but does not affect the variability of prediction performance.
Colombet
I
I
SPIM, Faculté de Médecine, 15 rue de l'école de Médecine, 75005 Paris, France. colombet@hegp.bhdc.jussieu.fr
Jaulent
M C
MC
Degoulet
P
P
Chatellier
G
G
D019540:Area Under Curve; D002318:Cardiovascular Diseases; D006801:Humans; D016015:Logistic Models; D012372:ROC Curve; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods*