title: Reliability of evaluating hospital quality by colorectal surgical site infection type.
abstract: OBJECTIVE
  To determine whether risk-adjusted colorectal SSI rates are statistically reliable as hospital quality measures.
  
  
  BACKGROUND
  Policymakers use surgical site infections (SSI) for public reporting of hospital quality and pay-for-performance because they are a relatively common and costly cause of patient morbidity.
  
  
  METHODS
  Patients who underwent a colorectal procedure in 2009 were identified from the American College of Surgeons National Surgical Quality Improvement Program. We developed hierarchical multivariate logistic models for (1) superficial SSI, (2) deep/organ-space SSI, and (3) "any SSI" and compared how each model ranked hospital-level risk-adjusted performance. Statistical reliability of hospital quality measurements was estimated on a scale from 0 to 1; with 0 indicating that apparent variation between a hospital's quality measurement and the average hospital is statistically unreliable, and 1 indicating that any observed variation is due to a real difference in performance.
  
  
  RESULTS
  Mean reliability of hospital-level quality measurements was 0.650 for superficial, 0.404 for deep/organ-space, and 0.586 for "any SSI." Lower reliability was accounted for by relatively little variation in risk-adjusted SSI rates between hospitals and insufficient numbers of colorectal cases submitted by individual hospitals. In 2009, we estimate that 22.1% of all US hospitals performed a sufficient number of colorectal cases to report superficial SSI rates at a high standard of statistical reliability and 1.0% did for deep/organ-space SSI.
  
  
  CONCLUSIONS
  As currently constructed, colorectal SSI quality measures might not meet a high standard of statistical reliability for most hospitals, limiting their ability to confidently differentiate high and low performance. Despite an expectation of improving statistical power, combining superficial and deep/organ-space SSI into an "any SSI" measure worsens reliability.
authors: *Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles †Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL ‡VA Greater Los Angeles Healthcare System, Los Angeles, California §RAND Corporation, Santa Monica, CA; and ¶Department of Surgery, Washington University School of Medicine in St Louis, Barnes Jewish Hospital, Saint Louis Veterans Affairs Medical Center, and Washington University Center for Health Policy and Olin Business School, St Louis, MO.
authors: Elise H
authors: EH
authors: Lawson
authors: Clifford Y
authors: CY
authors: Ko
authors: John L
authors: JL
authors: Adams
authors: Warren B
authors: WB
authors: Chow
authors: Bruce Lee
authors: BL
authors: Hall
mesh_terms: D003106:Colon / Q000601:surgery*; D005260:Female; D006761:Hospitals / Q000592:standards*; D006801:Humans; D008297:Male; D008875:Middle Aged; D011785:Quality Assurance, Health Care; D012007:Rectum / Q000601:surgery*; D015203:Reproducibility of Results; D013530:Surgical Wound Infection / Q000453:epidemiology*