title: Limitations of ranking lists based on cardiac surgery mortality rates.
abstract: BACKGROUND
  Ranking lists are a common way of reporting performance in cardiac surgery; however, rankings have shown to be imprecise, yet the extent of this imprecision is unknown. We aimed to determine the precision of, and fluctuations in, ranking lists in the comparison of cardiac surgery mortality rates.
  
  
  RESULTS
  Information on all adult cardiac surgery patients in all 16 cardiothoracic centers in The Netherlands from January 1, 2007, until December 31, 2009, was extracted from the database of the Netherlands Association for Cardio-Thoracic Surgery (n=46883). Ranks were assessed using crude and adjusted mortality rates, using a random effects logistic regression model. Risk adjustment was performed using the logistic EuroSCORE. Statistical precision of ranks was assessed with 95% confidence intervals. Additional analyses were performed for patients with isolated coronary artery bypass grafting. The ranking lists, based on mortality rates in 3 consecutive years, showed considerable reshuffling. When all data were pooled, the mean width of the 95% confidence intervals was 10 ranks using crude and 8 ranks using adjusted mortality rates. The large overlap of the confidence intervals across hospitals indicates that rank statistics were not materially different. Results were similar in the isolated coronary artery bypass grafting subgroup.
  
  
  CONCLUSIONS
  Rankings are an imprecise statistical method to report cardiac surgery mortality rates and prone to (random) fluctuation. Hence, reshuffling of ranks can be expected solely due to chance. Therefore, we strongly discourage the use of ranking lists in the comparison of mortality rates.
authors.affiliation: Department of Cardio-Thoracic Surgery, University Medical Center Utrecht, Utrecht, the Netherlands. s.siregar@umcutrecht.nl
authors.forename: Sabrina
authors.initials: S
authors.lastname: Siregar
authors.forename: Rolf H H
authors.initials: RH
authors.lastname: Groenwold
authors.forename: Evert K
authors.initials: EK
authors.lastname: Jansen
authors.forename: Michiel L
authors.initials: ML
authors.lastname: Bots
authors.forename: Yolanda
authors.initials: Y
authors.lastname: van der Graaf
authors.forename: Lex A
authors.initials: LA
authors.lastname: van Herwerden
mesh_terms: D006348:Cardiac Surgical Procedures / Q000401:mortality* / Q000639:trends*; D016001:Confidence Intervals; D003627:Data Interpretation, Statistical; D006761:Hospitals / Q000639:trends; D006801:Humans; D016015:Logistic Models; D009026:Mortality / Q000639:trends; D009426:Netherlands; D010043:Outcome and Process Assessment, Health Care / Q000639:trends*; D019984:Quality Indicators, Health Care / Q000639:trends*; D015203:Reproducibility of Results; D018570:Risk Assessment; D012307:Risk Factors; D013997:Time Factors; D016896:Treatment Outcome