title: Marginal likelihood estimation of negative binomial parameters with applications to RNA-seq data.
abstract: RNA-Seq data characteristically exhibits large variances, which need to be appropriately accounted for in any proposed model. We first explore the effects of this variability on the maximum likelihood estimator (MLE) of the dispersion parameter of the negative binomial distribution, and propose instead to use an estimator obtained via maximization of the marginal likelihood in a conjugate Bayesian framework. We show, via simulation studies, that the marginal MLE can better control this variation and produce a more stable and reliable estimator. We then formulate a conjugate Bayesian hierarchical model, and use this new estimator to propose a Bayesian hypothesis test to detect differentially expressed genes in RNA-Seq data. We use numerical studies to show that our much simpler approach is competitive with other negative binomial based procedures, and we use a real data set to illustrate the implementation and flexibility of the procedure.
authors.affiliation: Department of Biostatistics, University of Texas Health Science Center at Houston - School of Public Health, Houston, TX 77030, USA.
authors.forename: Luis
authors.initials: L
authors.lastname: León-Novelo
authors.affiliation: Department of of Statistics, Oregon State University, Corvallis, OR 97331, USA.
authors.forename: Claudio
authors.initials: C
authors.lastname: Fuentes
authors.affiliation: Department of of Statistics, Oregon State University, Corvallis, OR 97331, USA.
authors.forename: Sarah
authors.initials: S
authors.lastname: Emerson
mesh_terms: D016010:Binomial Distribution; D006801:Humans; D016013:Likelihood Functions; D017423:Sequence Analysis, RNA / Q000379:methods*
keywords: Bayesian methods; DEG; Hierarchical models; Hypothesis testing; Maximum likelihood estimation; Model selection; Negative binomial; RNA-Seq analysis