title: Identifying unusual performance in Australian and New Zealand intensive care units from 2000 to 2010.
abstract: BACKGROUND
  The Australian and New Zealand Intensive Care Society (ANZICS) Adult Patient Database (APD) collects voluntary data on patient admissions to Australian and New Zealand intensive care units (ICUs). This paper presents an in-depth statistical analysis of risk-adjusted mortality of ICU admissions from 2000 to 2010 for the purpose of identifying ICUs with unusual performance.
  
  
  METHODS
  A cohort of 523,462 patients from 144 ICUs was analysed. For each ICU, the natural logarithm of the standardised mortality ratio (log-SMR) was estimated from a risk-adjusted, three-level hierarchical model. This is the first time a three-level model has been fitted to such a large ICU database anywhere. The analysis was conducted in three stages which included the estimation of a null distribution to describe usual ICU performance. Log-SMRs with appropriate estimates of standard errors are presented in a funnel plot using 5% false discovery rate thresholds. False coverage-statement rate confidence intervals are also presented. The observed numbers of deaths for ICUs identified as unusual are compared to the predicted true worst numbers of deaths under the model for usual ICU performance.
  
  
  RESULTS
  Seven ICUs were identified as performing unusually over the period 2000 to 2010, in particular, demonstrating high risk-adjusted mortality compared to the majority of ICUs. Four of the seven were ICUs in private hospitals. Our three-stage approach to the analysis detected outlying ICUs which were not identified in a conventional (single) risk-adjusted model for mortality using SMRs to compare ICUs. We also observed a significant linear decline in mortality over the decade. Distinct yearly and weekly respiratory seasonal effects were observed across regions of Australia and New Zealand for the first time.
  
  
  CONCLUSIONS
  The statistical approach proposed in this paper is intended to be used for the review of observed ICU and hospital mortality. Two important messages from our study are firstly, that comprehensive risk-adjustment is essential in modelling patient mortality for comparing performance, and secondly, that the appropriate statistical analysis is complicated.
authors.affiliation: School of Mathematical Sciences, University of Adelaide, North Terrace, 5005 Adelaide, Australia. patty.solomon@adelaide.edu.au.
authors.forename: Patricia J
authors.initials: PJ
authors.lastname: Solomon
authors.forename: Jessica
authors.initials: J
authors.lastname: Kasza
authors.forename: John L
authors.initials: JL
authors.lastname: Moran
mesh_terms: D000704:Analysis of Variance; D001315:Australia; D016208:Databases, Factual; D005260:Female; D017052:Hospital Mortality; D006801:Humans; D007362:Intensive Care Units / Q000706:statistics & numerical data*; D008297:Male; D008499:Medical Records / Q000706:statistics & numerical data; D008875:Middle Aged; D015233:Models, Statistical; D009520:New Zealand; D010343:Patient Admission / Q000706:statistics & numerical data*; D012306:Risk; D012621:Seasons; D012720:Severity of Illness Index
references.citation: J Clin Epidemiol. 2013 Sep;66(9):1022-8
references.pmid: 23790725
references.citation: J Crit Care. 2006 Jun;21(2):133-41
references.pmid: 16769456
references.citation: Stat Med. 1997 Dec 15;16(23):2645-64
references.pmid: 9421867
references.citation: Stat Med. 1995 Oct 15;14(19):2161-72
references.pmid: 8552894
references.citation: J R Soc Med. 2012 Feb;105(2):74-84
references.pmid: 22307037
references.citation: J Eval Clin Pract. 2011 Feb;17(1):45-60
references.pmid: 20807296
references.citation: Med J Aust. 2011 Jun 20;194(12):658-60
references.pmid: 21692729
references.citation: Stat Med. 2013 Sep 20;32(21):3720-36
references.pmid: 23526209
references.citation: Stat Med. 2013 Jan 15;32(1):67-80
references.pmid: 22833304
references.citation: JAMA. 1997 Nov 19;278(19):1600-7
references.pmid: 9370507
references.citation: Biostatistics. 2013 Jul;14(3):477-90
references.pmid: 23376427
references.citation: Stat Med. 2007 Dec 30;26(30):5512-28
references.pmid: 18058845
references.citation: Med J Aust. 2011 Mar 21;194(6):287-92
references.pmid: 21426282
references.citation: Crit Care. 2006 Feb;10(1):103
references.pmid: 16420653
references.citation: BMC Med Res Methodol. 2014 Apr 22;14:53
references.pmid: 24755369
references.citation: Intensive Care Med. 2004 Oct;30(10):1900-7
references.pmid: 15300367
references.citation: Chest. 1991 Dec;100(6):1619-36
references.pmid: 1959406
references.citation: BMJ Qual Saf. 2013 Oct;22(10):870-6
references.pmid: 23832924
references.citation: Crit Care Med. 1985 Oct;13(10):818-29
references.pmid: 3928249
references.citation: JAMA. 1993 Dec 22-29;270(24):2957-63
references.pmid: 8254858
references.citation: PLoS One. 2013 Apr 09;8(4):e59160
references.pmid: 23593133
references.citation: Crit Care Med. 2008 Jan;36(1):46-61
references.pmid: 18090383
references.citation: J Epidemiol Community Health. 1999 Apr;53(4):235-8
references.pmid: 10396550