title: Validation of a risk stratification index and risk quantification index for predicting patient outcomes: in-hospital mortality, 30-day mortality, 1-year mortality, and length-of-stay.
abstract: BACKGROUND
  External validation of published risk stratification models is essential to determine their generalizability. This study evaluates the performance of the Risk Stratification Indices (RSIs) and 30-day mortality Risk Quantification Index (RQI).
  
  
  METHODS
  108,423 adult hospital admissions with anesthetics were identified (2006-2011). RSIs for mortality and length-of-stay endpoints were calculated using published methodology. 91,128 adult, noncardiac inpatient surgeries were identified with administrative data required for RQI calculation.
  
  
  RESULTS
  RSI in-hospital mortality and RQI 30-day mortality Brier scores were 0.308 and 0.017, respectively. RSI discrimination, by area under the receiver operating curves, was excellent at 0.966 (95% CI, 0.963-0.970) for in-hospital mortality, 0.903 (0.896-0.909) for 30-day mortality, 0.866 (0.861-0.870) for 1-yr mortality, and 0.884 (0.882-0.886) for length-of-stay. RSI calibration, however, was poor overall (17% predicted in-hospital mortality vs. 1.5% observed after inclusion of the regression constant) as demonstrated by calibration plots. Removal of self-fulfilling diagnosis and procedure codes (20,001 of 108,423; 20%) yielded similar results. RQIs were calculated for only 62,640 of 91,128 patients (68.7%) due to unmatched procedure codes. Patients with unmatched codes were younger, had higher American Society of Anesthesiologists physical status and 30-day mortality. The area under the receiver operating curve for 30-day mortality RQI was 0.888 (0.879-0.897). The model also demonstrated good calibration. Performance of a restricted index, Procedure Severity Score + American Society of Anesthesiologists physical status, performed as well as the original RQI model (age + American Society of Anesthesiologists + Procedure Severity Score).
  
  
  CONCLUSIONS
  Although the RSIs demonstrated excellent discrimination, poor calibration limits their generalizability. The 30-day mortality RQI performed well with age providing a limited contribution.
authors.affiliation: Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts 02114, USA. msigakis@partners.org
authors.forename: Matthew J G
authors.initials: MJ
authors.lastname: Sigakis
authors.forename: Edward A
authors.initials: EA
authors.lastname: Bittner
authors.forename: Jonathan P
authors.initials: JP
authors.lastname: Wanderer
mesh_terms: D000328:Adult; D000368:Aged; D019540:Area Under Curve; D005260:Female; D017052:Hospital Mortality; D006801:Humans; D007902:Length of Stay; D008297:Male; D008875:Middle Aged; D012372:ROC Curve; D012306:Risk; D013997:Time Factors; D016896:Treatment Outcome