title: Assessing Dissimilarity Measures for Sample-Based Hierarchical Clustering of RNA Sequencing Data Using Plasmode Datasets.
abstract: Sample- and gene-based hierarchical cluster analyses have been widely adopted as tools for exploring gene expression data in high-throughput experiments. Gene expression values (read counts) generated by RNA sequencing technology (RNA-seq) are discrete variables with special statistical properties, such as over-dispersion and right-skewness. Additionally, read counts are subject to technology artifacts as differences in sequencing depth. This possesses a challenge to finding distance measures suitable for hierarchical clustering. Normalization and transformation procedures have been proposed to favor the use of Euclidean and correlation based distances. Additionally, novel model-based dissimilarities that account for RNA-seq data characteristics have also been proposed. Adequacy of dissimilarity measures has been assessed using parametric simulations or exemplar datasets that may limit the scope of the conclusions. Here, we propose the simulation of realistic conditions through creation of plasmode datasets, to assess the adequacy of dissimilarity measures for sample-based hierarchical clustering of RNA-seq data. Consistent results were obtained using plasmode datasets based on RNA-seq experiments conducted under widely different conditions. Dissimilarity measures based on Euclidean distance that only considered data normalization or data standardization were not reliable to represent the expected hierarchical structure. Conversely, using either a Poisson-based dissimilarity or a rank correlation based dissimilarity or an appropriate data transformation, resulted in dendrograms that resemble the expected hierarchical structure. Plasmode datasets can be generated for a wide range of scenarios upon which dissimilarity measures can be evaluated for sample-based hierarchical clustering analysis. We showed different ways of generating such plasmodes and applied them to the problem of selecting a suitable dissimilarity measure. We report several measures that are satisfactory and the choice of a particular measure may rely on the availability on the software pipeline of preference.
authors.affiliation: Department of Fisheries and Wildlife, Michigan State University, East Lansing, Michigan, United States of America; Department of Statistics, Universidad Nacional del Comahue, Cinco Saltos, Rio Negro, Argentina.
authors.forename: Pablo D
authors.initials: PD
authors.lastname: Reeb
authors.affiliation: Department of Statistics, Universidad Nacional del Comahue, Cinco Saltos, Rio Negro, Argentina; College of Agricultural and Forest Sciences, Universidad Nacional de La Plata, La Plata, Buenos Aires, Argentina.
authors.forename: Sergio J
authors.initials: SJ
authors.lastname: Bramardi
authors.affiliation: Department of Fisheries and Wildlife, Michigan State University, East Lansing, Michigan, United States of America; Department of Animal Science, Michigan State University, East Lansing, Michigan, United States of America.
authors.forename: Juan P
authors.initials: JP
authors.lastname: Steibel
mesh_terms: D000465:Algorithms; D000818:Animals; D016000:Cluster Analysis; D030541:Databases, Genetic; D005260:Female; D008810:Mice, Inbred C57BL; D010802:Phylogeny; D017423:Sequence Analysis, RNA / Q000379:methods*; D034421:Sus scrofa