title: Model based heritability scores for high-throughput sequencing data.
abstract: BACKGROUND
  Heritability of a phenotypic or molecular trait measures the proportion of variance that is attributable to genotypic variance. It is an important concept in breeding and genetics. Few methods are available for calculating heritability for traits derived from high-throughput sequencing.
  
  
  RESULTS
  We propose several statistical models and different methods to compute and test a heritability measure for such data based on linear and generalized linear mixed effects models. We also provide methodology for hypothesis testing and interval estimation. Our analyses show that, among the methods, the negative binomial mixed model (NB-fit), compound Poisson mixed model (CP-fit), and the variance stabilizing transformed linear mixed model (VST) outperform the voom-transformed linear mixed model (voom). NB-fit and VST appear to be more robust than CP-fit for estimating and testing the heritability scores, while NB-fit is the most computationally expensive. CP-fit performed best in terms of the coverage of the confidence intervals. In addition, we applied the methods to both microRNA (miRNA) and messenger RNA (mRNA) sequencing datasets from a recombinant inbred mouse panel. We show that miRNA and mRNA expression can be a highly heritable molecular trait in mouse, and that some top heritable features coincide with expression quantitative trait loci.
  
  
  CONCLUSIONS
  The models and methods we investigated in this manuscript is applicable and extendable to sequencing experiments where some biological replicates are available and the environmental variation is properly controlled. The CP-fit approach for assessing heritability was implemented for the first time to our knowledge. All the methods presented, as well as the generation of simulated sequencing data under either negative binomial or compound Poisson mixed models, are provided in the R package HeritSeq.
authors.affiliation: Department of Biostatistics and Informatics, University of Colorado School of Public Health, Aurora, CO 80045, USA.
authors.forename: Pratyaydipta
authors.initials: P
authors.lastname: Rudra
authors.affiliation: Computational Bioscience Program, University of Colorado School of Medicine, Aurora, CO 80045, USA.
authors.forename: W Jenny
authors.initials: WJ
authors.lastname: Shi
authors.affiliation: Department of Biostatistics and Informatics, University of Colorado School of Public Health, Aurora, CO 80045, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Vestal
authors.affiliation: Department of Biostatistics and Informatics, University of Colorado School of Public Health, Aurora, CO 80045, USA.
authors.forename: Pamela H
authors.initials: PH
authors.lastname: Russell
authors.affiliation: Department of Biology, University of Oregon, Eugene, OR, USA.
authors.forename: Aaron
authors.initials: A
authors.lastname: Odell
authors.affiliation: Department of Molecular, Cellular and Developmental Biology, University of Colorado at Boulder, Boulder, CO 80303, USA.|BioFrontiers Institute, Boulder, CO 80303, USA.
authors.forename: Robin D
authors.initials: RD
authors.lastname: Dowell
authors.affiliation: Department of Pharmaceutical Sciences, University of Colorado Skaggs School of Pharmaceutical Sciences, Aurora, CO 80045, USA.
authors.forename: Richard A
authors.initials: RA
authors.lastname: Radcliffe
authors.affiliation: Department of Pharmaceutical Sciences, University of Colorado Skaggs School of Pharmaceutical Sciences, Aurora, CO 80045, USA.
authors.forename: Laura M
authors.initials: LM
authors.lastname: Saba
authors.affiliation: Department of Biostatistics and Informatics, University of Colorado School of Public Health, Aurora, CO 80045, USA. katerina.kechris@ucdenver.edu.
authors.forename: Katerina
authors.initials: K
authors.lastname: Kechris
mesh_terms: D000818:Animals; D005838:Genotype; D059014:High-Throughput Nucleotide Sequencing; D016014:Linear Models; D051379:Mice; D035683:MicroRNAs / Q000737:chemistry / Q000378:metabolism; D008957:Models, Genetic; D010641:Phenotype; D019655:Quantitative Trait, Heritable; D012333:RNA, Messenger / Q000737:chemistry / Q000378:metabolism; D017423:Sequence Analysis, RNA
chemical_list: D035683:MicroRNAs; D012333:RNA, Messenger
keywords: Compound Poisson mixed model; Heritability; Negative binomial mixed model; RNAseq; Recombinant inbred panel; Variance partition coefficient