title: Comparing Model Selection and Regularization Approaches to Variable Selection in Model-Based Clustering.
abstract: We compare two major approaches to variable selection in clustering: model selection and regularization. Based on previous results, we select the method of Maugis et al. (2009b), which modified the method of Raftery and Dean (2006), as a current state of the art model selection method. We select the method of Witten and Tibshirani (2010) as a current state of the art regularization method. We compared the methods by simulation in terms of their accuracy in both classification and variable selection. In the first simulation experiment all the variables were conditionally independent given cluster membership. We found that variable selection (of either kind) yielded substantial gains in classification accuracy when the clusters were well separated, but few gains when the clusters were close together. We found that the two variable selection methods had comparable classification accuracy, but that the model selection approach had substantially better accuracy in selecting variables. In our second simulation experiment, there were correlations among the variables given the cluster memberships. We found that the model selection approach was substantially more accurate in terms of both classification and variable selection than the regularization approach, and that both gave more accurate classifications thanK-means without variable selection. But the model selection approach is not available in a very high dimension context.
authors.affiliation: Inria Saclay - Île-de-France, Orsay, France. gilles.celeux@inria.fr.
authors.forename: Gilles
authors.initials: G
authors.lastname: Celeux
authors.affiliation: UMR INRA 1165 - UEVE, ERL CNRS 8196, Unité de Recherche en Génomique Végétale UMR INRA 1165-UEVE ERL CNRS 8196, Evry, France. UMR AgroParisTech/INRA MIA 518, Paris, France. marie_laure.martin@agroparistech.fr.
authors.forename: Marie-Laure
authors.initials: ML
authors.lastname: Martin-Magniette
authors.affiliation: Institut de Mathématiques de Toulouse, INSA de Toulouse, Université de Toulouse. cathy.maugis@insa-toulouse.fr.
authors.forename: Cathy
authors.initials: C
authors.lastname: Maugis-Rabusseau
authors.affiliation: Department of Statistics, University of Washington, Seattle, Washington, USA, and School of Mathematical Sciences, University College Dublin, Ireland. raftery@u.washington.edu.
authors.forename: Adrian E
authors.initials: AE
authors.lastname: Raftery
keywords: Model selection; Model-based clustering; Regularization approach; Variable selection