Predicted risk of mortality models: surgeons need to understand limitations of the University HealthSystem Consortium models.
BACKGROUND
The University HealthSystem Consortium (UHC) mortality risk adjustment models are increasingly being used as benchmarks for quality assessment. But these administrative database models may include postoperative complications in their adjustments for preoperative risk. The purpose of this study was to compare the performance of the UHC with the Society of Thoracic Surgeons (STS) risk-adjusted mortality models for adult cardiac surgery and evaluate the contribution of postoperative complications on model performance.


METHODS
We identified adult cardiac surgery patients with mortality risk estimates in both the UHC and Society of Thoracic Surgeons databases. We compared the predictive performance and calibration of estimates from both models. We then reestimated both models using only patients without any postoperative complications to determine the relative contribution of adjustments for postoperative events on model performance.


RESULTS
In the study population of 2,171 patients, the UHC model explained more variability (27% versus 13%, p < 0.001) and achieved better discrimination (C statistic = 0.88 versus 0.81, p < 0.001). But when applied in the population of patients without complications, the UHC model performance declined severely. The C statistic decreased from 0.88 to 0.49, a level of discrimination equivalent to random chance. The discrimination of the Society of Thoracic Surgeons model was unchanged (C statistic of 0.79 versus 0.81).


CONCLUSIONS
Although the UHC model demonstrated better performance in the total study population, this difference in performance reflects adjustments for conditions that are postoperative complications. The current UHC models should not be used for quality benchmarks.
Kozower
Benjamin D
BD
Department of Surgery, University of Virginia Health System, Charlottesville, VA, USA.
Ailawadi
Gorav
G
Jones
David R
DR
Pates
Robert D
RD
Lau
Christine L
CL
Kron
Irving L
IL
Stukenborg
George J
GJ
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D006348:Cardiac Surgical Procedures / Q000009:adverse effects / Q000401:mortality*; D001026:Coronary Artery Bypass / Q000401:mortality; D016208:Databases, Factual; D005260:Female; D019918:Heart Valve Prosthesis Implantation / Q000401:mortality; D006801:Humans; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D011237:Predictive Value of Tests; D012189:Retrospective Studies; D018570:Risk Assessment; D012307:Risk Factors; D014481:United States / Q000453:epidemiology