title: Effect of provider volume on the accuracy of hospital report cards: a Monte Carlo study.
abstract: BACKGROUND
  Hospital report cards, in which outcomes after the provision of medical or surgical care are compared across healthcare providers, are being published with increasing frequency. However, the accuracy of such comparisons is controversial, especially when case volumes are small. The objective was to determine the relationship between hospital case volume and the accuracy of hospital report cards.
  
  
  RESULTS
  Monte Carlo simulations were used to examine the influence of hospital case volume on the accuracy of hospital report cards in a setting in which true hospital performance was known with certainty, and perfect risk-adjustment was feasible. The parameters used to generate the simulated data sets were obtained from empirical analyses of data on patients hospitalized with acute myocardial infarction in Ontario, Canada, in which the overall 30-day mortality rate was 11.1%. We found that provider volume had a strong effect on the accuracy of hospital report cards. However, provider volume had to be >300 before ≥70% of hospitals were correctly classified. Furthermore, hospital volume had to be >1000 before ≥80% of hospitals were correctly classified.
  
  
  CONCLUSIONS
  Producers and users of hospital report cards need to be aware that, even when perfect risk adjustment is possible, the accuracy of hospital report cards is, at best, modest for small to medium-sized case loads (i.e., 100-300). Hospital report cards displayed high degrees of accuracy only when provider volumes exceeded the typical annual hospital case load for many cardiovascular conditions and procedures.
authors.affiliation: Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada.
authors.forename: Peter C
authors.initials: PC
authors.lastname: Austin
authors.forename: Mathew J
authors.initials: MJ
authors.lastname: Reeves
mesh_terms: D002170:Canada; D006282:Health Personnel; D006760:Hospitalization; D006761:Hospitals / Q000592:standards / Q000706:statistics & numerical data*; D006801:Humans; D009010:Monte Carlo Method; D009203:Myocardial Infarction / Q000453:epidemiology* / Q000401:mortality; D011785:Quality Assurance, Health Care / Q000706:statistics & numerical data*; D058028:Research Report; D020379:Risk Adjustment
keywords: health services research; risk adjustment