Equally valid models gave divergent predictions for mortality in acute myocardial infarction patients in a comparison of logistic [corrected] regression models.
OBJECTIVE
Models that predict mortality after acute myocardial infarction (AMI) contain different predictors and are based on different populations. We studied the agreement and validity of predictions for individual patients.


METHODS
We compared predictions from five predictive logistic regression models for short-term mortality after AMI. Three models were developed previously, and two models were developed in the GUSTO-I data, where all five models were applied (n =40,830, 7.0% 30-day mortality). Agreement was studied with weighted kappa statistics of categorized predictions. Validity was assessed by comparing observed frequencies with predictions (indicating calibration) and by the area under the receiver operating characteristic curve (AUC), indicating discriminative ability.


RESULTS
The predictions from the five models varied considerably for individual patients, with low agreement between most (kappa <0.6). Risk predictions from the three previously developed models were on average too high, which could be corrected by re-calibration of the model intercept. The AUC ranged from 0.76-0.78 and increased to 0.78-0.79 with re-estimated regression coefficients that were optimal for the GUSTO-I patients. The two more detailed GUSTO-I based models performed better (AUC approximately 0.82).


CONCLUSIONS
Models with different predictors may have a similar validity while the agreement between predictions for individual patients is poor. The main concerns in the applicability of predictive models for AMI should relate to the selected predictors and average calibration.
Steyerberg
Ewout W
EW
Department of Public Health, Center for Clinical Decision Sciences, Ee2093, Erasmus MC, University Medical Center Rotterdam, PO Box 1738, 3000 DR Rotterdam, The Netherlands. E.Steyerberg@ErasmusMC.nl
Eijkemans
Marinus J C
MJ
Boersma
Eric
E
Habbema
J D F
JD
D000368:Aged; D005260:Female; D006801:Humans; D016015:Logistic Models; D008297:Male; D009203:Myocardial Infarction / Q000401:mortality*; D011379:Prognosis; D015203:Reproducibility of Results; D012307:Risk Factors