title: What is the best way to estimate hospital quality outcomes? A simulation approach.
abstract: OBJECTIVE
  To test the accuracy of alternative estimators of hospital mortality quality using a Monte Carlo simulation experiment.
  
  
  METHODS
  Data are simulated to create an admission-level analytic dataset. The simulated data are validated by comparing distributional parameters (e.g., mean and standard deviation of 30-day mortality rate, hospital sample size) with the same parameters observed in Medicare data for acute myocardial infarction (AMI) inpatient admissions.
  
  
  METHODS
  We perform a Monte Carlo simulation experiment in which true quality is known to test the accuracy of the Observed-over-Expected estimator, the Risk Standardized Mortality Rate (RSMR), the Dimick and Staiger (DS) estimator, the Hierarchical Poisson estimator, and the Moving Average estimator using hospital 30-day mortality for AMI as the outcome. Estimator accuracy is evaluated for all hospitals and for small, medium, and large hospitals.
  
  
  METHODS
  Data are simulated.
  
  
  RESULTS
  Significant and substantial variation is observed in the accuracy of the tested outcome estimators. The DS estimator is the most accurate for all hospitals and for small hospitals using both accuracy criteria (root mean squared error and proportion of hospitals correctly classified into quintiles).
  
  
  CONCLUSIONS
  The mortality estimator currently in use by Medicare for public quality reporting, the RSMR, has been shown to be less accurate than the DS estimator, although the magnitude of the difference is not large. Pending testing and validation of our findings using current hospital data, CMS should reconsider the decision to publicly report mortality rates using the RSMR.
authors.affiliation: Weill Cornell Medical College, Department of Public Health, Division of Outcomes and Effectiveness, New York, NY 10065, USA. amr2015@med.cornell.edu
authors.forename: Andrew
authors.initials: A
authors.lastname: Ryan
authors.forename: James
authors.initials: J
authors.lastname: Burgess
authors.forename: Robert
authors.initials: R
authors.lastname: Strawderman
authors.forename: Justin
authors.initials: J
authors.lastname: Dimick
mesh_terms: D003625:Data Collection / Q000379:methods; D006302:Health Services Research; D017052:Hospital Mortality; D006761:Hospitals / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D006278:Medicare; D009010:Monte Carlo Method; D009203:Myocardial Infarction / Q000401:mortality*; D010043:Outcome and Process Assessment, Health Care / Q000592:standards; D016012:Poisson Distribution; D019984:Quality Indicators, Health Care; D011787:Quality of Health Care; D018570:Risk Assessment; D014481:United States / Q000453:epidemiology