title: Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes.
abstract: BACKGROUND
  Sepsis is the focus of national quality improvement programs and a recent public reporting measure from the Centers for Medicare and Medicaid Services. However, diagnosing sepsis requires interpreting nonspecific signs and can therefore be subjective. We sought to quantify interobserver variability in diagnosing sepsis.
  
  
  METHODS
  We distributed five case vignettes of patients with suspected or confirmed infection and organ dysfunction to a sample of practicing intensivists. Respondents classified cases as systemic inflammatory response syndrome, sepsis, severe sepsis, septic shock, or none of the above. Interobserver variability was calculated using Fleiss' κ for the five-level classification, and for answers dichotomized as severe sepsis/septic shock versus not-severe sepsis/septic shock and any sepsis category (sepsis, severe sepsis, or septic shock) versus not-sepsis.
  
  
  RESULTS
  Ninety-four physicians completed the survey. Most respondents (88%) identified as critical care specialists; other specialties included pulmonology (39%), anesthesia (19%), surgery (9%), and emergency medicine (9%). Respondents had been in practice for a median of 8 years, and 90% practiced at academic hospitals. Almost all respondents (83%) felt strongly or somewhat confident in their ability to apply the traditional consensus sepsis definitions. However, overall interrater agreement in sepsis diagnoses was poor (Fleiss' κ 0.29). When responses were dichotomized into severe sepsis/septic shock versus not-severe sepsis/septic shock or any sepsis category versus not-sepsis, agreement was still poor (Fleiss' κ 0.23 and 0.18, respectively). Seventeen percent of respondents classified one of the five cases as severe sepsis/septic shock, 27.7% rated two cases, 33.0% respondents rated three cases, 19.2% rated four cases, and 3.2% rated all five cases as severe sepsis/septic shock. Among respondents who felt strongly confident in their ability to use sepsis definitions (n = 45), agreement was no better (Fleiss' κ 0.28 for the five-category classification, and Fleiss' κ 0.21 for the dichotomized severe sepsis/septic shock classification). Cases were felt to be extremely or very realistic in 74% of responses; only 3% were deemed unrealistic.
  
  
  CONCLUSIONS
  Diagnosing sepsis is extremely subjective and variable. Objective criteria and standardized methodology are needed to enhance consistency and comparability in sepsis research, surveillance, benchmarking, and reporting.
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Drive, Suite 401, Boston, MA, USA. crhee1@partners.org.|Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA. crhee1@partners.org.
authors.forename: Chanu
authors.identifier: http://orcid.org/0000-0002-9537-4245
authors.initials: C
authors.lastname: Rhee
authors.affiliation: Critical Care Medicine Department, Clinical Center, National Institutes of Health, Bethesda, MD, USA.
authors.forename: Sameer S
authors.initials: SS
authors.lastname: Kadri
authors.affiliation: Critical Care Medicine Department, Clinical Center, National Institutes of Health, Bethesda, MD, USA.
authors.forename: Robert L
authors.initials: RL
authors.lastname: Danner
authors.affiliation: Critical Care Medicine Department, Clinical Center, National Institutes of Health, Bethesda, MD, USA.
authors.forename: Anthony F
authors.initials: AF
authors.lastname: Suffredini
authors.affiliation: Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA.
authors.forename: Anthony F
authors.initials: AF
authors.lastname: Massaro
authors.affiliation: Department of Medicine, North Shore Medical Center, Salem, MA, USA.
authors.forename: Barrett T
authors.initials: BT
authors.lastname: Kitch
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Drive, Suite 401, Boston, MA, USA.
authors.forename: Grace
authors.initials: G
authors.lastname: Lee
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Drive, Suite 401, Boston, MA, USA.|Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA.
authors.forename: Michael
authors.initials: M
authors.lastname: Klompas
mesh_terms: D003422:Critical Care / Q000379:methods / Q000592:standards*; D019091:Critical Pathways; D020000:Decision Support Systems, Clinical; D042241:Early Diagnosis; D005260:Female; D006801:Humans; D008297:Male; D058996:Quality Improvement; D018805:Sepsis / Q000175:diagnosis* / Q000628:therapy; D012720:Severity of Illness Index; D011795:Surveys and Questionnaires; D018746:Systemic Inflammatory Response Syndrome / Q000175:diagnosis; D014481:United States