Prediction rules for complications in coronary bypass surgery: a comparison and methodological critique.
BACKGROUND
Clinical prediction rules have been developed that use preoperative information to stratify patients according to risk of complications after cardiac surgery.


OBJECTIVE
To assess the methodological standards and performance of 7 models.


METHODS
The validation portion of the Quality Measurement and Management Initiative (QMMI) cohort included a random sample of all adult patients (n = 3,261) who underwent coronary artery bypass grafting (CABG) surgery not involving valvular or other concomitant procedures at 12 medical centers from August 1993 to October 1995.


METHODS
Methodological standards used for model comparison were adapted from published criteria. Model performance was assessed by receiver-operating characteristic (ROC) analysis, and calibration was evaluated with the Hosmer-Lemeshow (HL) statistic and observed-expected plots.


METHODS
We performed cross-validation by applying the published criteria for the development of each model to the validation subset of the QMMI cohort and by assessing the performance of each model in discriminating outcomes.


RESULTS
Wide variations existed in the methodologies used to develop and validate the 5 additive scores evaluated. Cross-validation of all 5 additive scores revealed degradation in their abilities to discriminate outcomes. The 2 logistic models examined performed similarly to the additive scores examined in predicting mortality.


CONCLUSIONS
Substantial variation existed both in the methodologies used to develop models and in the ability of the models to predict outcomes. Models developed at single institutions or using fewer patients may be less generalizable when applied to diverse clinical settings. Additive and logistic regression models performed similarly, as assessed by ROC and HL analyses.
Fortescue
E B
EB
Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Mass 02115, USA.
Kahn
K
K
Bates
D W
DW
D000328:Adult; D015331:Cohort Studies; D001026:Coronary Artery Bypass / Q000009:adverse effects*; D005260:Female; D017052:Hospital Mortality; D006801:Humans; D008297:Male; D015233:Models, Statistical; D015337:Multicenter Studies as Topic; D011183:Postoperative Complications; D011379:Prognosis; D019984:Quality Indicators, Health Care; D012372:ROC Curve; D012307:Risk Factors; D012720:Severity of Illness Index; D014481:United States