title: When policy gets it right: variability in u.s. Hospitals' diagnosis of ventilator-associated pneumonia*.
abstract: OBJECTIVE
  The Centers for Disease Control has recently proposed a major change in how ventilator-associated pneumonia is defined. This has profound implications for public reporting, reimbursement, and accountability measures for ICUs. We sought to provide evidence for or against this change by quantifying limitations of the national definition of ventilator-associated pneumonia that was in place until January 2013, particularly with regard to comparisons between, and ranking of, hospitals and ICUs.
  
  
  METHODS
  A prospective survey of a nationally representative group of 43 hospitals, randomly selected from the American Hospital Association Guide (2009). Subjects classified six standardized vignettes of possible cases of ventilator-associated pneumonia as pneumonia or no pneumonia.
  
  
  METHODS
  Individuals responsible for ventilator-associated pneumonia surveillance at 43 U.S. hospitals.
  
  
  METHODS
  None.
  
  
  RESULTS
  We measured the proportion of standardized cases classified as ventilator-associated pneumonia. Of 138 hospitals consented, 61 partially completed the survey and 43 fully completed the survey (response rate 44% and 31%, respectively). Agreement among hospitals about classification of cases as ventilator-associated pneumonia/not ventilator-associated pneumonia was nearly random (Fleiss κ 0.13). Some hospitals rated 0% of cases as having pneumonia; others classified 100% as having pneumonia (median, 50%; interquartile range, 33-66%). Although region of the country did not predict case assignment, respondents who described their region as "rural" were more likely to judge a case to be pneumonia than respondents elsewhere (relative risk, 1.25, Kruskal-Wallis chi-square, p = 0.03).
  
  
  CONCLUSIONS
  In this nationally representative study of hospitals, assignment of ventilator-associated pneumonia is extremely variable, enough to render comparisons between hospitals worthless, even when standardized cases eliminate variability in clinical data abstraction. The magnitude of this variability highlights the limitations of using poorly performing surveillance definitions as methods of hospital evaluation and comparison, and our study provides very strong support for moving to a more objective definition of ventilator-associated complications.
authors.affiliation: 1Department of Medicine, Division of Pulmonary, Critical Care, and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA. 2Center for Healthcare Delivery Science, Beth Israel Deaconess Medical Center, Boston, MA. 3Harvard Medical School, Boston, MA. 4Department of Medicine, Division of Infectious Diseases, Beth Israel Deaconess Medical Center, Boston, MA. 5Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Boston, MA. 6Section of Pulmonary and Critical Care, Department of Medicine, University of Chicago, Chicago, IL.
authors.forename: Jennifer P
authors.initials: JP
authors.lastname: Stevens
authors.forename: Bartlomiej
authors.initials: B
authors.lastname: Kachniarz
authors.forename: Sharon B
authors.initials: SB
authors.lastname: Wright
authors.forename: Jean
authors.initials: J
authors.lastname: Gillis
authors.forename: Daniel
authors.initials: D
authors.lastname: Talmor
authors.forename: Peter
authors.initials: P
authors.lastname: Clardy
authors.forename: Michael D
authors.initials: MD
authors.lastname: Howell
mesh_terms: D002487:Centers for Disease Control and Prevention, U.S. / Q000592:standards*; D016009:Chi-Square Distribution; D003430:Cross-Sectional Studies; D005260:Female; D006291:Health Policy; D006761:Hospitals / Q000592:standards*; D006801:Humans; D015994:Incidence; D007362:Intensive Care Units; D008297:Male; D015588:Observer Variation; D053717:Pneumonia, Ventilator-Associated / Q000175:diagnosis* / Q000453:epidemiology* / Q000628:therapy; D011050:Policy Making; D011446:Prospective Studies; D019984:Quality Indicators, Health Care; D014481:United States / Q000453:epidemiology