title: Statistical inference for time course RNA-Seq data using a negative binomial mixed-effect model.
abstract: BACKGROUND
  Accurate identification of differentially expressed (DE) genes in time course RNA-Seq data is crucial for understanding the dynamics of transcriptional regulatory network. However, most of the available methods treat gene expressions at different time points as replicates and test the significance of the mean expression difference between treatments or conditions irrespective of time. They thus fail to identify many DE genes with different profiles across time. In this article, we propose a negative binomial mixed-effect model (NBMM) to identify DE genes in time course RNA-Seq data. In the NBMM, mean gene expression is characterized by a fixed effect, and time dependency is described by random effects. The NBMM is very flexible and can be fitted to both unreplicated and replicated time course RNA-Seq data via a penalized likelihood method. By comparing gene expression profiles over time, we further classify the DE genes into two subtypes to enhance the understanding of expression dynamics. A significance test for detecting DE genes is derived using a Kullback-Leibler distance ratio. Additionally, a significance test for gene sets is developed using a gene set score.
  
  
  RESULTS
  Simulation analysis shows that the NBMM outperforms currently available methods for detecting DE genes and gene sets. Moreover, our real data analysis of fruit fly developmental time course RNA-Seq data demonstrates the NBMM identifies biologically relevant genes which are well justified by gene ontology analysis.
  
  
  CONCLUSIONS
  The proposed method is powerful and efficient to detect biologically relevant DE genes and gene sets in time course RNA-Seq data.
authors: Department of Statistics, University of Georgia, 101 Cedar Street, Athens, 30602, USA.
authors: Xiaoxiao
authors: X
authors: Sun
authors: Department of Statistics, University of Illinois at Urbana-Champaign, 725 South Wright Street, Champaign, 61820, USA.
authors: David
authors: D
authors: Dalpiaz
authors: Department of Statistics, Harvard University, One Oxford Street, Cambridge, 02138, USA.
authors: Di
authors: D
authors: Wu
authors: Department of Statistics, Harvard University, One Oxford Street, Cambridge, 02138, USA.
authors: Jun
authors: J
authors: S Liu
authors: Department of Statistics, University of Georgia, 101 Cedar Street, Athens, 30602, USA.
authors: Wenxuan
authors: W
authors: Zhong
authors: Department of Statistics, University of Georgia, 101 Cedar Street, Athens, 30602, USA. pingma@uga.edu.
authors: Ping
authors: P
authors: Ma
mesh_terms: D000818:Animals; D004330:Drosophila / Q000235:genetics / Q000378:metabolism; D016013:Likelihood Functions; D015233:Models, Statistical; D012313:RNA / Q000737:chemistry* / Q000378:metabolism; D017423:Sequence Analysis, RNA
chemical_list: D012313:RNA
keywords: Analysis of variance; Differentially expressed gene; Gene set enrichment; Penalized likelihood; Smoothing spline