title: Statistical evaluation of methods for identification of differentially abundant genes in comparative metagenomics.
abstract: BACKGROUND
  Metagenomics is the study of microbial communities by sequencing of genetic material directly from environmental or clinical samples. The genes present in the metagenomes are quantified by annotating and counting the generated DNA fragments. Identification of differentially abundant genes between metagenomes can provide important information about differences in community structure, diversity and biological function. Metagenomic data is however high-dimensional, contain high levels of biological and technical noise and have typically few biological replicates. The statistical analysis is therefore challenging and many approaches have been suggested to date.
  
  
  RESULTS
  In this article we perform a comprehensive evaluation of 14 methods for identification of differentially abundant genes between metagenomes. The methods are compared based on the power to detect differentially abundant genes and their ability to correctly estimate the type I error rate and the false discovery rate. We show that sample size, effect size, and gene abundance greatly affect the performance of all methods. Several of the methods also show non-optimal model assumptions and biased false discovery rate estimates, which can result in too large numbers of false positives. We also demonstrate that the performance of several of the methods differs substantially between metagenomic data sequenced by different technologies.
  
  
  CONCLUSIONS
  Two methods, primarily designed for the analysis of RNA sequencing data (edgeR and DESeq2) together with a generalized linear model based on an overdispersed Poisson distribution were found to have best overall performance. The results presented in this study may serve as a guide for selecting suitable statistical methods for identification of differentially abundant genes in metagenomes.
authors.affiliation: Department of Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, Gothenburg, SE-412 96, Sweden. viktor.jonsson@chalmers.se.
authors.forename: Viktor
authors.initials: V
authors.lastname: Jonsson
authors.affiliation: Department of Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, Gothenburg, SE-412 96, Sweden. tobias.osterlund@chalmers.se.
authors.forename: Tobias
authors.initials: T
authors.lastname: Österlund
authors.affiliation: Department of Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, Gothenburg, SE-412 96, Sweden. nerman@chalmers.se.
authors.forename: Olle
authors.initials: O
authors.lastname: Nerman
authors.affiliation: Department of Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, Gothenburg, SE-412 96, Sweden. erik.kristiansson@chalmers.se.
authors.forename: Erik
authors.initials: E
authors.lastname: Kristiansson
mesh_terms: D054892:Metagenome / Q000235:genetics; D056186:Metagenomics / Q000379:methods*; D017423:Sequence Analysis, RNA; D012984:Software
references.citation: BMC Bioinformatics. 2007;8:387
references.pmid: 17937807
references.citation: Bioinformatics. 2014 Nov 1;30(21):3123-4
references.pmid: 25061070
references.citation: Genome Res. 2002 Apr;12(4):656-64
references.pmid: 11932250
references.citation: Nat Biotechnol. 2012 Jun;30(6):513-20
references.pmid: 22678395
references.citation: Nature. 2012 Oct 4;490(7418):55-60
references.pmid: 23023125
references.citation: Bioinformatics. 2015 Jul 15;31(14):2269-75
references.pmid: 25792553
references.citation: Bioinformatics. 2010 Jan 1;26(1):139-40
references.pmid: 19910308
references.citation: PLoS Comput Biol. 2009 Apr;5(4):e1000352
references.pmid: 19360128
references.citation: BMC Bioinformatics. 2013;14:38
references.pmid: 23375020
references.citation: BMC Microbiol. 2015;15:66
references.pmid: 25880246
references.citation: Genome Biol. 2005;6(8):229
references.pmid: 16086859
references.citation: Nucleic Acids Res. 2012 Jan;40(Database issue):D123-9
references.pmid: 22086953
references.citation: PLoS One. 2015;10(4):e0124360
references.pmid: 25867897
references.citation: Genome Biol. 2010;11(10):R106
references.pmid: 20979621
references.citation: Biometrika. 2011 Mar;98(1):199-214
references.pmid: 23049127
references.citation: Gigascience. 2015 Sep 07;4:40
references.pmid: 26351566
references.citation: Nucleic Acids Res. 2012 Nov 1;40(20):10084-97
references.pmid: 22965124
references.citation: Appl Environ Microbiol. 2011 Apr;77(7):2513-21
references.pmid: 21317268
references.citation: Genome Biol. 2011;12(6):R60
references.pmid: 21702898
references.citation: Nucleic Acids Res. 2003 Jan 1;31(1):371-3
references.pmid: 12520025
references.citation: Proc Natl Acad Sci U S A. 2003 Aug 5;100(16):9440-5
references.pmid: 12883005
references.citation: PLoS One. 2014;9(6):e97338
references.pmid: 24921648
references.citation: Bioinformatics. 2009 Oct 15;25(20):2737-8
references.pmid: 19696045
references.citation: Genome Biol. 2010;11(3):R25
references.pmid: 20196867
references.citation: BMC Bioinformatics. 2013;14:91
references.pmid: 23497356
references.citation: Chem Biol. 1998 Oct;5(10):R245-9
references.pmid: 9818143
references.citation: Nucleic Acids Res. 2015 Sep 3;43(15):e97
references.pmid: 25925576
references.citation: Nucleic Acids Res. 2008 Jan;36(Database issue):D534-8
references.pmid: 17932063
references.citation: Genome Biol. 2014;15(2):R29
references.pmid: 24485249
references.citation: PLoS Comput Biol. 2010 Feb;6(2):e1000667
references.pmid: 20195499
references.citation: Environ Microbiol. 2015 Jul;17(7):2239-53
references.pmid: 25329041
references.citation: Nature. 2012 Jun 14;486(7402):222-7
references.pmid: 22699611
references.citation: PLoS Comput Biol. 2014 Apr;10(4):e1003531
references.pmid: 24699258
references.citation: Stat Appl Genet Mol Biol. 2004;3:Article3
references.pmid: 16646809
references.citation: BMC Bioinformatics. 2006;7:162
references.pmid: 16549025
references.citation: Bioinformatics. 2010 Mar 15;26(6):715-21
references.pmid: 20130030
references.citation: Nature. 2010 Mar 4;464(7285):59-65
references.pmid: 20203603
references.citation: Appl Environ Microbiol. 2011 Feb;77(4):1315-24
references.pmid: 21183646
references.citation: Nucleic Acids Res. 2012 May;40(10):4288-97
references.pmid: 22287627
references.citation: PLoS Comput Biol. 2011 Oct;7(10):e1002195
references.pmid: 22039361
references.citation: BMC Bioinformatics. 2008;9:386
references.pmid: 18803844
references.citation: PLoS Comput Biol. 2014 Jul;10(7):e1003706
references.pmid: 25010449
references.citation: Bioinformatics. 2007 Nov 1;23(21):2881-7
references.pmid: 17881408
references.citation: Genome Biol. 2013;14(2):202
references.pmid: 23514020
references.citation: Nat Methods. 2013 Dec;10(12):1200-2
references.pmid: 24076764
references.citation: Mol Ecol. 2014 Sep;23(18):4498-510
references.pmid: 25113243
references.citation: Nat Methods. 2014 Nov;11(11):1144-6
references.pmid: 25218180
references.citation: J Comput Sci Technol. 2009 Jan;25(1):71-81
references.pmid: 20648230
references.citation: Nature. 2013 Jun 6;498(7452):99-103
references.pmid: 23719380
references.citation: ISME J. 2010 Aug;4(8):1075-7
references.pmid: 20336158
references.citation: J Dairy Sci. 2013 Sep;96(9):6030-46
references.pmid: 23871375
references.citation: Appl Environ Microbiol. 2000 Jun;66(6):2541-7
references.pmid: 10831436
references.citation: Genome Biol. 2014;15(12):550
references.pmid: 25516281