Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example.
OBJECTIVE
There is growing interest in developing prediction models. The accuracy of such models when applied in new patient samples is commonly lower than estimated from the development sample. This may be because of differences between the samples and/or because the developed model was overfitted (too optimistic). Various methods, including bootstrapping techniques exist for afterwards shrinking the regression coefficients and the model's discrimination and calibration for overoptimism. Penalized maximum likelihood estimation (PMLE) is a more rigorous method because adjustment for overfitting is directly built into the model development, instead of relying on shrinkage afterwards. PMLE has been described mainly in the statistical literature and is rarely applied to empirical data. Using empirical data, we illustrate the use of PMLE to develop a prediction model.


METHODS
The accuracy of the final PMLE model will be contrasted with the final models derived by ordinary stepwise logistic regression without and with shrinkage afterwards. The potential advantages and disadvantages of PMLE over the other two strategies are discussed.


RESULTS
PMLE leads to smaller prediction errors, provides for model reduction to a user-defined degree, and may differently shrink each predictor for overoptimism without sacrificing much discriminative accuracy of the model.


CONCLUSIONS
PMLE is an easily applicable and promising method to directly adjust clinical prediction models for overoptimism.
Moons
K G M
KG
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 80035, 3508 TA Utrecht, The Netherlands. K.G.M.Moons@umcutrecht.nl
Donders
A Rogier T
AR
Steyerberg
E W
EW
Harrell
F E
FE
D015982:Bias; D005260:Female; D006801:Humans; D016013:Likelihood Functions; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D011379:Prognosis; D011655:Pulmonary Embolism / Q000175:diagnosis / Q000628:therapy; D016896:Treatment Outcome