title: Off the shelf or recalibrate? customizing a risk index for assessing mortality.
abstract: BACKGROUND
  Public "report cards" for cardiac surgery have been freely available from a variety of sources. These risk-adjusted indices serve as a means of benchmarking outcomes performances, allowing comparisons of outcomes between surgical programs, and quantifying quality improvement programs. We examined two alternative strategies for using previously developed risk-adjusted mortality models in a community hospital: (1) using the model "off the shelf" (OTS) and (2) recalibrating the existing model (RM) to fit the institution-specific population.
  
  
  METHODS
  Six OTS models were used: Parsonnet (PA), Canadian (CA), Cleveland (CL), Northern New England (NNE), New York (NY), and New Jersey (NJ). The RM models were created by each model's independent variables and definitions and adjusting the weighting with logistic regression methods. The accuracy, the C statistic, and the precision of each model were assessed for in-hospital mortality. We compared the OTS version of each model to the RM version with methods detailed by Hanley and McNeil.
  
  
  RESULTS
  The RM C statistic was improved for all risk-adjusted models, most notably in the statistical improvement seen in the PA (0.053 improvement) and NJ (0.052 improvement) indices. Statistical gains in precision were also seen in the RM models for the PA, CL, and NNE indices. Conversely, one model, the CA model, was more poorly calibrated in the RM model compared with the OTS model, despite an improved C statistic (0.062).
  
  
  CONCLUSIONS
  The RM strategy provides institution-explicit models that demonstrate a higher degree of accuracy and precision than the OTS models.
authors.affiliation: Research Institute, Division of Cardiovascular Surgery, St. Luke's Hospital Health Network, Bethlehem, Pennsylvania, USA. ReedJ@slhn.org
authors.forename: James F
authors.initials: JF
authors.lastname: Reed
authors.forename: Stephen A
authors.initials: SA
authors.lastname: Olenchock
authors.forename: Sabina A
authors.initials: SA
authors.lastname: Murphy
authors.forename: Fernando M
authors.initials: FM
authors.lastname: Garzia
mesh_terms: D000367:Age Factors; D000368:Aged; D000369:Aged, 80 and over; D000465:Algorithms; D001026:Coronary Artery Bypass / Q000401:mortality*; D005260:Female; D017052:Hospital Mortality; D006764:Hospitals, Community; D006801:Humans; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D008962:Models, Theoretical; D010043:Outcome and Process Assessment, Health Care; D020379:Risk Adjustment; D018570:Risk Assessment; D012307:Risk Factors; D012737:Sex Factors