Better-than-average and worse-than-average hospitals may not significantly differ from average hospitals: an analysis of Medicare Hospital Compare ratings.
BACKGROUND
Public report card designers aim to provide comprehensible provider performance information to consumers. Report cards often display classifications of providers into performance tiers that reflect whether performance is statistically significantly above or below average or not statistically significantly different from average. To further enhance the salience of public reporting to consumers, report card websites often allow a user to compare a subset of selected providers on tiered performance rather than direct statistical comparisons of the providers in a consumer's personal choice set.


OBJECTIVE
We illustrate the differences in conclusions drawn about relative provider performance using tiers versus conducing statistical tests to assess performance differences.


METHODS
Using publicly available cross-sectional data from Medicare Hospital Compare on three mortality and three readmission outcome measures, we compared each provider in the top or bottom performance tier with those in the middle tier and assessed the proportion of such comparisons that exhibited no statistically significant differences.


RESULTS
Across the six outcomes, 1.3-6.1% of hospitals were classified in the top tier. Each top-tier hospital did not statistically significantly differ in performance from at least one mid-tier hospital. The percentages of mid-tier hospitals that were not statistically significantly different from a given top-tier hospital were 74.3-81.1%. The percentages of hospitals classified as bottom tier were 0.6-4.0%. Each bottom-tier hospital showed no statistically significant difference from at least one mid-tier hospital. The percentage of mid-tier hospitals that were not significantly different from a bottom-tier hospital ranged from 60.4% to 74.8%.


CONCLUSIONS
Our analyses illustrate the need for further innovations in the design of public report cards to enhance their salience for consumers.
RAND Corporation, Santa Monica, California, USA.
Susan M
SM
Paddock
Kaiser Permanente Research, Pasadena, California, USA.
John L
JL
Adams
RAND Corporation, Santa Monica, California, USA.
Fernando
F
Hoces de la Guardia
D017052:Hospital Mortality; D006761:Hospitals / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D006278:Medicare / Q000592:standards; D010359:Patient Readmission / Q000706:statistics & numerical data; D019984:Quality Indicators, Health Care; D011787:Quality of Health Care / Q000592:standards / Q000706:statistics & numerical data; D014481:United States
Performance measures; Quality measurement; Report cards; Statistics