Predicted risk of mortality models: surgeons need to understand limitations of the University HealthSystem Consortium models.
BACKGROUND
The University HealthSystem Consortium (UHC) mortality risk adjustment models are increasingly being used as benchmarks for quality assessment. But these administrative database models may include postoperative complications in their adjustments for preoperative risk. The purpose of this study was to compare the performance of the UHC with the Society of Thoracic Surgeons (STS) risk-adjusted mortality models for adult cardiac surgery and evaluate the contribution of postoperative complications on model performance.


METHODS
We identified adult cardiac surgery patients with mortality risk estimates in both the UHC and Society of Thoracic Surgeons databases. We compared the predictive performance and calibration of estimates from both models. We then reestimated both models using only patients without any postoperative complications to determine the relative contribution of adjustments for postoperative events on model performance.


RESULTS
In the study population of 2,171 patients, the UHC model explained more variability (27% versus 13%, p < 0.001) and achieved better discrimination (C statistic = 0.88 versus 0.81, p < 0.001). But when applied in the population of patients without complications, the UHC model performance declined severely. The C statistic decreased from 0.88 to 0.49, a level of discrimination equivalent to random chance. The discrimination of the Society of Thoracic Surgeons model was unchanged (C statistic of 0.79 versus 0.81).


CONCLUSIONS
Although the UHC model demonstrated better performance in the total study population, this difference in performance reflects adjustments for conditions that are postoperative complications. The current UHC models should not be used for quality benchmarks.
Department of Surgery, University of Virginia Health System, Charlottesville, VA, USA.
Benjamin D
BD
Kozower
Gorav
G
Ailawadi
David R
DR
Jones
Robert D
RD
Pates
Christine L
CL
Lau
Irving L
IL
Kron
George J
GJ
Stukenborg
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D006348:Cardiac Surgical Procedures / Q000009:adverse effects / Q000401:mortality*; D001026:Coronary Artery Bypass / Q000401:mortality; D016208:Databases, Factual; D005260:Female; D019918:Heart Valve Prosthesis Implantation / Q000401:mortality; D006801:Humans; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D011237:Predictive Value of Tests; D012189:Retrospective Studies; D018570:Risk Assessment; D012307:Risk Factors; D014481:United States / Q000453:epidemiology