title: Benchmarking trauma centers on mortality alone does not reflect quality of care: implications for pay-for-performance.
abstract: BACKGROUND
  Trauma centers are currently benchmarked on mortality outcomes alone. However, pay-for-performance measures may financially penalize centers based on complications. Our objective was to determine whether the results would be similar to the current standard method of mortality-based benchmarking if trauma centers were profiled on complications.
  
  
  METHODS
  We analyzed data from the National Trauma Data Bank from 2007 to 2010. Patients 16 years or older with blunt or penetrating injuries and an Injury Severity Score (ISS) of 9 or higher were included. Risk-adjusted observed-to-expected (O/E) mortality ratios for each center were generated and used to rank each facility as high, average, or low performing. We similarly ranked facilities on O/E morbidity ratios defined as occurrence of any major complication. Concordance between hospital performance rankings was evaluated using a weighted κ statistic. Correlation between morbidity- and mortality-based O/E ratios was assessed using Pearson coefficients. Sensitivity analyses were performed to mitigate the competing risk of death for the morbidity analyses.
  
  
  RESULTS
  A total of 449,743 patients from 248 facilities were analyzed. The unadjusted morbidity and mortality rates were 10.0% and 6.9%, respectively. No correlation was found between morbidity- and mortality-based O/E ratios (r = -0.01). Only 40% of the centers had similar performance rankings for both mortality and morbidity. Of the 31 high performers for mortality, only 11 centers were also high performers for morbidity. A total of 78 centers were ranked as average, and 11 ranked as low performers on both outcomes. Comparison of hospital performance status using mortality and morbidity outcomes demonstrated poor concordance (weighted κ = 0.03, p = 0.22).
  
  
  CONCLUSIONS
  Mortality-based external benchmarking does not identify centers with high complication rates. This creates a dichotomy between current trauma center profiling standards and measures used for pay-for-performance. A benchmarking mechanism that reflects all measures of quality is needed.
  
  
  METHODS
  Prognostic/epidemiologic study, level III.
authors.affiliation: From the Center for Surgical Trials and Outcomes Research (Z.G.H., E.B.S., E.R.H., A.H.H.), and Division of Acute Care Surgery, Trauma, Emergency Surgery and Critical Care (E.R.H., A.H.H.), Department of Surgery, Department of Emergency Medicine (ERH), and Department of Anesthesiology and Critical Care Medicine (A.L.), The Johns Hopkins School of Medicine; and Department of Health Policy and Management (R.C., A.H.H.), Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland; and Department of Surgery (S.N.Z., E.C.C.), Howard University College of Medicine, Washington, District of Columbia.
authors.forename: Zain G
authors.initials: ZG
authors.lastname: Hashmi
authors.forename: Eric B
authors.initials: EB
authors.lastname: Schneider
authors.forename: Renan
authors.initials: R
authors.lastname: Castillo
authors.forename: Elliott R
authors.initials: ER
authors.lastname: Haut
authors.forename: Syed Nabeel
authors.initials: SN
authors.lastname: Zafar
authors.forename: Edward E
authors.initials: EE
authors.lastname: Cornwell
authors.forename: Ellen J
authors.initials: EJ
authors.lastname: Mackenzie
authors.forename: Asad
authors.initials: A
authors.lastname: Latif
authors.forename: Adil H
authors.initials: AH
authors.lastname: Haider
mesh_terms: D000328:Adult; D000368:Aged; D019985:Benchmarking / Q000379:methods*; D002423:Cause of Death; D016208:Databases, Factual; D005260:Female; D019538:Health Care Surveys; D017052:Hospital Mortality / Q000639:trends*; D006801:Humans; D015601:Injury Severity Score; D008297:Male; D008875:Middle Aged; D011787:Quality of Health Care; D012052:Reimbursement, Incentive / Q000191:economics*; D020379:Risk Adjustment; D012680:Sensitivity and Specificity; D016019:Survival Analysis; D014193:Trauma Centers / Q000191:economics / Q000592:standards* / Q000706:statistics & numerical data*; D014481:United States; D014947:Wounds and Injuries / Q000175:diagnosis / Q000401:mortality* / Q000628:therapy; D014949:Wounds, Nonpenetrating / Q000175:diagnosis / Q000401:mortality / Q000628:therapy; D014950:Wounds, Penetrating / Q000175:diagnosis / Q000401:mortality / Q000628:therapy; D055815:Young Adult