title: The impact of statistical choices on neonatal intensive care unit quality ratings based on nosocomial infection rates.
abstract: OBJECTIVE
  To examine the extent to which performance assessment methods affect the percentage of neonatal intensive care units (NICUs) and very low-birth-weight (VLBW) infants included in performance assessments, the distribution of NICU performance ratings, and the level of agreement in those ratings.
  
  
  METHODS
  Cross-sectional study based on risk-adjusted nosocomial infection rates.
  
  
  METHODS
  NICUs belonging to the California Perinatal Quality Care Collaborative 2007-2008.
  
  
  METHODS
  One hundred twenty-six California NICUs and 10 487 VLBW infants.
  
  
  METHODS
  Three performance assessment choices: (1) excluding "low-volume" NICUs (those caring for <30 VLBW infants per year) vs a criterion based on confidence intervals, (2) using Bayesian vs frequentist hierarchical models, and (3) pooling data across 1 vs 2 years.
  
  
  METHODS
  Proportion of NICUs and patients included in quality assessment, distribution of ratings for NICUs, and agreement between methods using the κ statistic.
  
  
  RESULTS
  Depending on the methods applied, 51% to 85% of NICUs and 72% to 96% of VLBW infants were included in performance assessments, 76% to 87% of NICUs were considered "average," and the level of agreement between NICU ratings ranged from 0.23 to 0.89.
  
  
  CONCLUSIONS
  The percentage of NICUs included in performance assessments and their ratings can shift dramatically depending on performance measurement method. Physicians, payers, and policymakers should continue to closely examine which existing performance assessment methods are most appropriate for evaluating pediatric care quality.
authors.affiliation: Department of Pediatrics, Division of Neonatology, University of California at San Francisco, 533 Parnassus Avenue, San Francisco, CA 94143, USA. LeeHC@peds.ucsf.edu
authors.forename: Henry C
authors.initials: HC
authors.lastname: Lee
authors.forename: Alyna T
authors.initials: AT
authors.lastname: Chien
authors.forename: Naomi S
authors.initials: NS
authors.lastname: Bardach
authors.forename: Ted
authors.initials: T
authors.lastname: Clay
authors.forename: Jeffrey B
authors.initials: JB
authors.lastname: Gould
authors.forename: R Adams
authors.initials: RA
authors.lastname: Dudley
mesh_terms: D002140:California; D003428:Cross Infection / Q000175:diagnosis / Q000453:epidemiology* / Q000628:therapy; D003430:Cross-Sectional Studies; D005260:Female; D017052:Hospital Mortality / Q000639:trends*; D006801:Humans; D015994:Incidence; D007224:Infant Care / Q000458:organization & administration; D007231:Infant, Newborn; D019102:Infant, Very Low Birth Weight; D007363:Intensive Care Units, Neonatal / Q000592:standards* / Q000706:statistics & numerical data; D008297:Male; D017063:Outcome Assessment, Health Care; D010348:Patient Care Team / Q000458:organization & administration*; D018570:Risk Assessment; D012720:Severity of Illness Index; D013647:Task Performance and Analysis; D017749:Total Quality Management