Aggregating published prediction models with individual participant data: a comparison of different approaches.
During the recent decades, interest in prediction models has substantially increased, but approaches to synthesize evidence from previously developed models have failed to keep pace. This causes researchers to ignore potentially useful past evidence when developing a novel prediction model with individual participant data (IPD) from their population of interest. We aimed to evaluate approaches to aggregate previously published prediction models with new data. We consider the situation that models are reported in the literature with predictors similar to those available in an IPD dataset. We adopt a two-stage method and explore three approaches to calculate a synthesis model, hereby relying on the principles of multivariate meta-analysis. The former approach employs a naive pooling strategy, whereas the latter accounts for within-study and between-study covariance. These approaches are applied to a collection of 15 datasets of patients with traumatic brain injury, and to five previously published models for predicting deep venous thrombosis. Here, we illustrated how the generally unrealistic assumption of consistency in the availability of evidence across included studies can be relaxed. Results from the case studies demonstrate that aggregation yields prediction models with an improved discrimination and calibration in a vast majority of scenarios, and result in equivalent performance (compared with the standard approach) in a small minority of situations. The proposed aggregation approaches are particularly useful when few participant data are at hand. Assessing the degree of heterogeneity between IPD and literature findings remains crucial to determine the optimal approach in aggregating previous evidence into new prediction models.
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands. T.Debray@umcutrecht.nl
Thomas P A
TP
Debray
Hendrik
H
Koffijberg
Yvonne
Y
Vergouwe
Karel G M
KG
Moons
Ewout W
EW
Steyerberg
D001930:Brain Injuries / Q000473:pathology; D003627:Data Interpretation, Statistical; D005544:Forecasting; D023261:Glasgow Outcome Scale; D006801:Humans; D015233:Models, Statistical; D015999:Multivariate Analysis; D020246:Venous Thrombosis / Q000473:pathology