Effect of choice of estimation method on inter-hospital mortality rate comparisons.
OBJECTIVE
To evaluate and compare the use of 6 different methods for calculating expected mortality rates and standardized mortality ratios (SMRs) when performing interhospital mortality rate comparisons.


METHODS
Retrospective cohort study using actual and simulated hospitalization data to evaluate the use of (1) fixed-effects, (2) Generalized Linear Mixed Model, and (3) Bayesian (Markov Chain Monte Carlo-based) random-effects models on both aggregated and individual-level data to estimate SMRs by hospital.


METHODS
Seventeen hospitals in a large integrated health care delivery system.


METHODS
Inpatient mortality.


RESULTS
Results from the 6 different methods compared in this study were highly correlated both on log(SMR) values and hospital ranks (range, 0.91-1). All the methods had high specificity (>87%) for finding true underlying mortality effects. The fixed-effects models had higher overall sensitivity than the random-effects models. The individual-level random effects model had generally higher sensitivity than the aggregated random-effects models. All methods showed a high correlation with the true ranks.


CONCLUSIONS
When comparing mortality rates across hospitals, it is important to focus not only on the method used to measure patient sickness but also on the analytical technique used to estimate hospital-specific adjusted mortality rates. When an illness severity measure including detailed physiologic data was used, the simplest method we examined, a fixed-effects aggregate-level approach in common usage, out-performed the other methods when both specificity and sensitivity are considered. Use of a severity measure that correlates less well with mortality than the one we employed would be expected to reduce the sensitivity of all of the methods examined.
Kipnis
Patricia
P
Kaiser Foundation Health Plan, Management, Information and Analysis, Oakland, CA 94612, USA. Patricia.Kipnis@kp.org
Escobar
Gabriel J
GJ
Draper
David
D
D019985:Benchmarking / Q000379:methods; D015331:Cohort Studies; D017052:Hospital Mortality; D006801:Humans; D019984:Quality Indicators, Health Care; D015203:Reproducibility of Results; D012189:Retrospective Studies; D020379:Risk Adjustment; D012720:Severity of Illness Index