Off the shelf or recalibrate? customizing a risk index for assessing mortality.
BACKGROUND
Public "report cards" for cardiac surgery have been freely available from a variety of sources. These risk-adjusted indices serve as a means of benchmarking outcomes performances, allowing comparisons of outcomes between surgical programs, and quantifying quality improvement programs. We examined two alternative strategies for using previously developed risk-adjusted mortality models in a community hospital: (1) using the model "off the shelf" (OTS) and (2) recalibrating the existing model (RM) to fit the institution-specific population.


METHODS
Six OTS models were used: Parsonnet (PA), Canadian (CA), Cleveland (CL), Northern New England (NNE), New York (NY), and New Jersey (NJ). The RM models were created by each model's independent variables and definitions and adjusting the weighting with logistic regression methods. The accuracy, the C statistic, and the precision of each model were assessed for in-hospital mortality. We compared the OTS version of each model to the RM version with methods detailed by Hanley and McNeil.


RESULTS
The RM C statistic was improved for all risk-adjusted models, most notably in the statistical improvement seen in the PA (0.053 improvement) and NJ (0.052 improvement) indices. Statistical gains in precision were also seen in the RM models for the PA, CL, and NNE indices. Conversely, one model, the CA model, was more poorly calibrated in the RM model compared with the OTS model, despite an improved C statistic (0.062).


CONCLUSIONS
The RM strategy provides institution-explicit models that demonstrate a higher degree of accuracy and precision than the OTS models.
Reed
James F
JF
Research Institute, Division of Cardiovascular Surgery, St. Luke's Hospital Health Network, Bethlehem, Pennsylvania, USA. ReedJ@slhn.org
Olenchock
Stephen A
SA
Murphy
Sabina A
SA
Garzia
Fernando M
FM
D000367:Age Factors; D000368:Aged; D000369:Aged, 80 and over; D000465:Algorithms; D001026:Coronary Artery Bypass / Q000401:mortality*; D005260:Female; D017052:Hospital Mortality; D006764:Hospitals, Community; D006801:Humans; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D008962:Models, Theoretical; D010043:Outcome and Process Assessment, Health Care; D020379:Risk Adjustment; D018570:Risk Assessment; D012307:Risk Factors; D012737:Sex Factors