Interobserver agreement of Centers for Disease Control and Prevention criteria for classifying infections in critically ill patients.
OBJECTIVE
Correct classification of the source of infection is important in observational and interventional studies of sepsis. Centers for Disease Control and Prevention criteria are most commonly used for this purpose, but the robustness of these definitions in critically ill patients is not known. We hypothesized that in a mixed ICU population, the performance of these criteria would be generally reduced and would vary among diagnostic subgroups.


METHODS
Prospective cohort.


METHODS
Data were collected as part of a cohort of 1,214 critically ill patients admitted to two hospitals in The Netherlands between January 2011 and June 2011.


METHODS
Eight observers assessed a random sample of 168 of 554 patients who had experienced at least one infectious episode in the ICU. Each patient was assessed by two randomly selected observers who independently scored the source of infection (by affected organ system or site), the plausibility of infection (rated as none, possible, probable, or definite), and the most likely causative pathogen. Assessments were based on a post hoc review of all available clinical, radiological, and microbiological evidence. The observed diagnostic agreement for source of infection was classified as partial (i.e., matching on organ system or site) or complete (i.e., matching on specific diagnostic terms), for plausibility as partial (2-point scale) or complete (4-point scale), and for causative pathogens as an approximate or exact pathogen match. Interobserver agreement was expressed as a concordant percentage and as a kappa statistic.


METHODS
None.


RESULTS
A total of 206 infectious episodes were observed. Agreement regarding the source of infection was 89% (183/206) and 69% (142/206) for a partial and complete diagnostic match, respectively. This resulted in a kappa of 0.85 (95% CI, 0.79-0.90). Agreement varied from 63% to 91% within major diagnostic categories and from 35% to 97% within specific diagnostic subgroups, with the lowest concordance observed in cases of ventilator-associated pneumonia. In the 142 episodes for which a complete match on source of infection was obtained, the interobserver agreement for plausibility of infection was 83% and 65% on a 2- and 4-point scale, respectively. For causative pathogen, agreement was 78% and 70% for an approximate and exact pathogen match, respectively.


CONCLUSIONS
Interobserver agreement for classifying sources of infection using Centers for Disease Control and Prevention criteria was excellent overall. However, full concordance on all aspects of the diagnosis between independent observers was rare for some types of infection, in particular for ventilator-associated pneumonia.
1Department of Medical Microbiology, University Medical Center Utrecht, Utrecht, The Netherlands. 2Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands. 3Department of Intensive Care, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. 4Center of Experimental and Molecular Medicine & Division of Infectious Diseases, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. 5Department of Intensive Care, University Medical Center Utrecht, Utrecht, The Netherlands.
Peter M C
PM
Klein Klouwenberg
David S Y
DS
Ong
Lieuwe D J
LD
Bos
Friso M
FM
de Beer
Roosmarijn T M
RT
van Hooijdonk
Mischa A
MA
Huson
Marleen
M
Straat
Lonneke A
LA
van Vught
Luuk
L
Wieske
Janneke
J
Horn
Marcus J
MJ
Schultz
Tom
T
van der Poll
Marc J M
MJ
Bonten
Olaf L
OL
Cremer
D000368:Aged; D002487:Centers for Disease Control and Prevention, U.S. / Q000592:standards*; D016001:Confidence Intervals; D016638:Critical Illness; D003428:Cross Infection / Q000453:epidemiology* / Q000209:etiology*; D005260:Female; D006801:Humans; D007362:Intensive Care Units; D008297:Male; D008875:Middle Aged; D009426:Netherlands / Q000453:epidemiology; D015588:Observer Variation; D011446:Prospective Studies; D014481:United States