Benchmarking Sepsis Gene Expression Diagnostics Using Public Data.
OBJECTIVE
In response to a need for better sepsis diagnostics, several new gene expression classifiers have been recently published, including the 11-gene "Sepsis MetaScore," the "FAIM3-to-PLAC8" ratio, and the Septicyte Lab. We performed a systematic search for publicly available gene expression data in sepsis and tested each gene expression classifier in all included datasets. We also created a public repository of sepsis gene expression data to encourage their future reuse.


METHODS
We searched National Institutes of Health Gene Expression Omnibus and EBI ArrayExpress for human gene expression microarray datasets. We also included the Glue Grant trauma gene expression cohorts.


METHODS
We selected clinical, time-matched, whole blood studies of sepsis and acute infections as compared to healthy and/or noninfectious inflammation patients. We identified 39 datasets composed of 3,241 samples from 2,604 patients.


METHODS
All data were renormalized from raw data, when available, using consistent methods.


RESULTS
Mean validation areas under the receiver operating characteristic curve for discriminating septic patients from patients with noninfectious inflammation for the Sepsis MetaScore, the FAIM3-to-PLAC8 ratio, and the Septicyte Lab were 0.82 (range, 0.73-0.89), 0.78 (range, 0.49-0.96), and 0.73 (range, 0.44-0.90), respectively. Paired-sample t tests of validation datasets showed no significant differences in area under the receiver operating characteristic curves. Mean validation area under the receiver operating characteristic curves for discriminating infected patients from healthy controls for the Sepsis MetaScore, FAIM3-to-PLAC8 ratio, and Septicyte Lab were 0.97 (range, 0.85-1.0), 0.94 (range, 0.65-1.0), and 0.71 (range, 0.24-1.0), respectively. There were few significant differences in any diagnostics due to pathogen type.


CONCLUSIONS
The three diagnostics do not show significant differences in overall ability to distinguish noninfectious systemic inflammatory response syndrome from sepsis, though the performance in some datasets was low (area under the receiver operating characteristic curve, < 0.7) for the FAIM3-to-PLAC8 ratio and Septicyte Lab. The Septicyte Lab also demonstrated significantly worse performance in discriminating infections as compared to healthy controls. Overall, public gene expression data are a useful tool for benchmarking gene expression diagnostics.
1Stanford Institute for Immunity, Transplantation and Infection, Stanford University School of Medicine, Stanford, CA. 2Division of Biomedical Informatics Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA.
Timothy E
TE
Sweeney
Purvesh
P
Khatri
D019985:Benchmarking; D016022:Case-Control Studies; D066264:Datasets as Topic; D003937:Diagnosis, Differential; D015870:Gene Expression; D006801:Humans; D012680:Sensitivity and Specificity; D018805:Sepsis / Q000175:diagnosis* / Q000235:genetics*; D018746:Systemic Inflammatory Response Syndrome / Q000175:diagnosis
Nature. 2010 Aug 19;466(7309):973-7
20725040
Crit Care Med. 2010 Apr;38(4):1045-53
20048677
PLoS Negl Trop Dis. 2015 Mar 13;9(3):e0003522
25768297
BMC Res Notes. 2012 Aug 02;5:401
22852767
Physiol Genomics. 2008 Jun 12;34(1):127-34
18460642
Crit Care. 2011 Jun 20;15(3):R149
21682927
Nat Commun. 2014 Aug 14;5:4649
25120092
Mol Med. 2007 Sep-Oct;13(9-10):495-508
17932561
PLoS Med. 2013 Nov;10(11):e1001549
24265599
PLoS One. 2011 Mar 08;6(3):e17186
21408152
Am J Respir Crit Care Med. 2015 Nov 15;192(10):1261-2
26568248
Intensive Care Med Exp. 2014 Dec;2(1):20
26215705
BMC Bioinformatics. 2008 Aug 12;9 Suppl 9:S10
18793455
Proc Natl Acad Sci U S A. 2013 Jul 30;110(31):12792-7
23858444
Lancet Infect Dis. 2015 May;15(5):581-614
25932591
Sci Transl Med. 2015 May 13;7(287):287ra71
25972003
J Immunol. 2015 Oct 1;195(7):3248-3261
26311902
Cell Host Microbe. 2009 Sep 17;6(3):207-17
19664979
Am J Respir Crit Care Med. 2015 Nov 15;192(10 ):1260-1
26568247
J Infect. 2011 Oct;63(4):308-16
21781987
Genome Biol. 2009;10(11):R127
19903332
Crit Care Med. 2009 May;37(5):1558-66
19325468
Cell Host Microbe. 2014 Jul 9;16(1):115-27
24981333
PLoS One. 2013;8(1):e48979
23326304
Am J Respir Crit Care Med. 2015 Oct 1;192(7):826-35
26121490
PLoS Med. 2015 Dec 08;12(12):e1001916
26645559
PLoS Pathog. 2015 Jun 12;11(6):e1004869
26070066
PLoS Med. 2008 Sep 30;5(9):e184
18767902
Physiol Genomics. 2007 Jul 18;30(2):146-55
17374846
Nat Genet. 2001 Nov;29(3):306-9
11600885
Future Microbiol. 2008 Oct;3(5):563-78
18811240
J Infect Dis. 2015 Jul 15;212(2):213-22
25637350
J Infect Dis. 2013 Nov 15;208(10):1664-8
23901082
J Exp Med. 2011 Dec 19;208(13):2581-90
22110166
BMC Med Genomics. 2012 May 04;5:13
22559298
Mol Med. 2011;17(11-12):1146-56
21738952
Crit Care Med. 2014 Aug;42(8):1749-55
24717459
Mol Med. 2009 Jul-Aug;15(7-8):220-7
19593405
Crit Care. 2012 Aug 16;16(4):R157
22898401
Crit Care. 2010;14(5):R167
20840779
Am J Respir Crit Care Med. 2012 Jun 1;185(11):1225-34
22461369