title: Reliability of Pressure Ulcer Rates: How Precisely Can We Differentiate Among Hospital Units, and Does the Standard Signal-Noise Reliability Measure Reflect This Precision?
abstract: Hospital performance reports often include rankings of unit pressure ulcer rates. Differentiating among units on the basis of quality requires reliable measurement. Our objectives were to describe and apply methods for assessing reliability of hospital-acquired pressure ulcer rates and evaluate a standard signal-noise reliability measure as an indicator of precision of differentiation among units. Quarterly pressure ulcer data from 8,199 critical care, step-down, medical, surgical, and medical-surgical nursing units from 1,299 US hospitals were analyzed. Using beta-binomial models, we estimated between-unit variability (signal) and within-unit variability (noise) in annual unit pressure ulcer rates. Signal-noise reliability was computed as the ratio of between-unit variability to the total of between- and within-unit variability. To assess precision of differentiation among units based on ranked pressure ulcer rates, we simulated data to estimate the probabilities of a unit's observed pressure ulcer rate rank in a given sample falling within five and ten percentiles of its true rank, and the probabilities of units with ulcer rates in the highest quartile and highest decile being identified as such. We assessed the signal-noise measure as an indicator of differentiation precision by computing its correlations with these probabilities. Pressure ulcer rates based on a single year of quarterly or weekly prevalence surveys were too susceptible to noise to allow for precise differentiation among units, and signal-noise reliability was a poor indicator of precision of differentiation. To ensure precise differentiation on the basis of true differences, alternative methods of assessing reliability should be applied to measures purported to differentiate among providers or units based on quality. © 2016 The Authors. Research in Nursing & Health published by Wiley Periodicals, Inc.
authors.affiliation: Health Services and Outcomes Research, Children's Mercy Hospitals and Clinics, School of Medicine, University of Missouri-Kansas City, 2401 Gillham Road, Kansas City, MO, 64108.
authors.forename: Vincent S
authors.initials: VS
authors.lastname: Staggs
authors.affiliation: School of Nursing, University of Kansas Medical Center, Kansas City, KS.
authors.forename: Emily
authors.initials: E
authors.lastname: Cramer
mesh_terms: D006757:Hospital Units / Q000706:statistics & numerical data*; D006801:Humans; D015233:Models, Statistical; D009741:Nursing Staff, Hospital; D003668:Pressure Ulcer / Q000453:epidemiology*; D015995:Prevalence; D019984:Quality Indicators, Health Care; D015203:Reproducibility of Results; D018570:Risk Assessment
keywords: healthcare quality; patient safety; pressure ulcers; quality measurement; reliability