title: Statistical significance for hierarchical clustering.
abstract: Cluster analysis has proved to be an invaluable tool for the exploratory and unsupervised analysis of high-dimensional datasets. Among methods for clustering, hierarchical approaches have enjoyed substantial popularity in genomics and other fields for their ability to simultaneously uncover multiple layers of clustering structure. A critical and challenging question in cluster analysis is whether the identified clusters represent important underlying structure or are artifacts of natural sampling variation. Few approaches have been proposed for addressing this problem in the context of hierarchical clustering, for which the problem is further complicated by the natural tree structure of the partition, and the multiplicity of tests required to parse the layers of nested clusters. In this article, we propose a Monte Carlo based approach for testing statistical significance in hierarchical clustering which addresses these issues. The approach is implemented as a sequential testing procedure guaranteeing control of the family-wise error rate. Theoretical justification is provided for our approach, and its power to detect true clustering structure is illustrated through several simulation studies and applications to two cancer gene expression datasets.
authors.affiliation: Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, North Carolina, U.S.A.
authors.forename: Patrick K
authors.initials: PK
authors.lastname: Kimes
authors.affiliation: Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, North Carolina, U.S.A.|Department of Biostatistics, University of North Carolina at Chapel Hill, North Carolina, U.S.A.|Department of Genetics, University of North Carolina at Chapel Hill, North Carolina, U.S.A.|Carolina Center for Genome Sciences, University of North Carolina at Chapel Hill, North Carolina, U.S.A.|Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, North Carolina, U.S.A.
authors.forename: Yufeng
authors.initials: Y
authors.lastname: Liu
authors.affiliation: Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, North Carolina, U.S.A.
authors.forename: David
authors.initials: D
authors.lastname: Neil Hayes
authors.affiliation: Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, North Carolina, U.S.A.|Department of Biostatistics, University of North Carolina at Chapel Hill, North Carolina, U.S.A.|Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, North Carolina, U.S.A.
authors.forename: James Stephen
authors.initials: JS
authors.lastname: Marron
mesh_terms: D000465:Algorithms; D016000:Cluster Analysis; D023281:Genomics; D006801:Humans
keywords: High-dimension; Hypothesis testing; Multiple correction; Unsupervised learning