title: A benchmark for evaluation of algorithms for identification of cellular correlates of clinical outcomes.
abstract: The Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) challenges were established to compare the performance of computational methods for identifying cell populations in multidimensional flow cytometry data. Here we report the results of FlowCAP-IV where algorithms from seven different research groups predicted the time to progression to AIDS among a cohort of 384 HIV+ subjects, using antigen-stimulated peripheral blood mononuclear cell (PBMC) samples analyzed with a 14-color staining panel. Two approaches (FlowReMi.1 and flowDensity-flowType-RchyOptimyx) provided statistically significant predictive value in the blinded test set. Manual validation of submitted results indicated that unbiased analysis of single cell phenotypes could reveal unexpected cell types that correlated with outcomes of interest in high dimensional flow cytometry datasets.
authors.affiliation: Terry Fox Laboratory, British Columbia Cancer Agency, Vancouver, British Columbia, Canada.|Department of Medical Genetics, University of British Columbia, Vancouver, British Columbia, Canada.|Baxter Laboratory in Stem Cell Biology, Stanford University, Stanford, California.
authors.forename: Nima
authors.initials: N
authors.lastname: Aghaeepour
authors.affiliation: ImmunoTechnology Section, Vaccine Research Center, National Institutes of Health, Washington, DC.
authors.forename: Pratip
authors.initials: P
authors.lastname: Chattopadhyay
authors.affiliation: Department Computational and Systems Biology, University of Pittsburgh, Pittsburg, Pennsylvania.
authors.forename: Maria
authors.initials: M
authors.lastname: Chikina
authors.affiliation: Department of Information Technology, Ghent University-iMinds, Ghent, Belgium.
authors.forename: Tom
authors.initials: T
authors.lastname: Dhaene
authors.affiliation: Department of Information Technology, Ghent University-iMinds, Ghent, Belgium.|Inflammation Research Center, VIB, Ghent, Belgium.|Department of Respiratory Medicine, Ghent University Hospital, Ghent, Belgium.
authors.forename: Sofie
authors.initials: S
authors.lastname: Van Gassen
authors.affiliation: Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland.
authors.forename: Miron
authors.initials: M
authors.lastname: Kursa
authors.affiliation: Inflammation Research Center, VIB, Ghent, Belgium.|Department of Respiratory Medicine, Ghent University Hospital, Ghent, Belgium.
authors.forename: Bart N
authors.initials: BN
authors.lastname: Lambrecht
authors.affiliation: Terry Fox Laboratory, British Columbia Cancer Agency, Vancouver, British Columbia, Canada.
authors.forename: Mehrnoush
authors.initials: M
authors.lastname: Malek
authors.affiliation: Department of Statistics, University of Queensland, St. Lucia, Brisbane, Australia 4072.
authors.forename: G J
authors.initials: GJ
authors.lastname: McLachlan
authors.affiliation: J. Craig Venter Institute, La Jolla, California.
authors.forename: Yu
authors.initials: Y
authors.lastname: Qian
authors.affiliation: Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, Georgia.
authors.forename: Peng
authors.initials: P
authors.lastname: Qiu
authors.affiliation: Inflammation Research Center, VIB, Ghent, Belgium.|Department of Respiratory Medicine, Ghent University Hospital, Ghent, Belgium.
authors.forename: Yvan
authors.initials: Y
authors.lastname: Saeys
authors.affiliation: J. Craig Venter Institute, La Jolla, California.
authors.forename: Rick
authors.initials: R
authors.lastname: Stanton
authors.affiliation: The John Van Geest Cancer Research Centre, Nottingham Trent University, Nottingham, United Kingdom.|Centre for Genomic and Experimental Medicine, Institute of Genetics and Molecular Medicine, University of Edinburgh, Edinburgh, United Kingdom.
authors.forename: Dong
authors.initials: D
authors.lastname: Tong
authors.affiliation: Inflammation Research Center, VIB, Ghent, Belgium.|Department of Respiratory Medicine, Ghent University Hospital, Ghent, Belgium.|Department of Public Health and Primary Care, kU Leuven Kulak, Kortrijk, Belgium.
authors.forename: Celine
authors.initials: C
authors.lastname: Vens
authors.affiliation: Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland.
authors.forename: Sławomir
authors.initials: S
authors.lastname: Walkowiak
authors.affiliation: Department of Mathematics, University of Queensland, St. Lucia, Brisbane, Australia.|School of Medicine, Shihezi University, Xinjiang, 832000, China.
authors.forename: Kui
authors.initials: K
authors.lastname: Wang
authors.affiliation: Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, DC.
authors.forename: Greg
authors.initials: G
authors.lastname: Finak
authors.affiliation: Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, DC.
authors.forename: Raphael
authors.initials: R
authors.lastname: Gottardo
authors.affiliation: David H. Smith Center for Vaccine Biology and Immunology, University of Rochester Medical Center, Rochester, New York.
authors.forename: Tim
authors.initials: T
authors.lastname: Mosmann
authors.affiliation: Baxter Laboratory in Stem Cell Biology, Stanford University, Stanford, California.
authors.forename: Garry P
authors.initials: GP
authors.lastname: Nolan
authors.affiliation: J. Craig Venter Institute, La Jolla, California.|Department of Pathology, University of California, San Diego, California.
authors.forename: Richard H
authors.initials: RH
authors.lastname: Scheuermann
authors.affiliation: Terry Fox Laboratory, British Columbia Cancer Agency, Vancouver, British Columbia, Canada.|Department of Medical Genetics, University of British Columbia, Vancouver, British Columbia, Canada.
authors.forename: Ryan R
authors.initials: RR
authors.lastname: Brinkman
mesh_terms: D000163:Acquired Immunodeficiency Syndrome / Q000175:diagnosis / Q000473:pathology*; D000465:Algorithms; D019985:Benchmarking; D019295:Computational Biology / Q000379:methods*; D003627:Data Interpretation, Statistical; D018450:Disease Progression; D005434:Flow Cytometry / Q000379:methods*; D006679:HIV Seropositivity; D006801:Humans; D013194:Staining and Labeling; D013601:T-Lymphocytes / Q000166:cytology*
keywords: HIV; bioinformatics; classification; clinical outcome; clustering; data analysis; flow cytometry; supervised analysis