title: A unified statistical approach to non-negative matrix factorization and probabilistic latent semantic indexing.
abstract: Non-negative matrix factorization (NMF) is a powerful machine learning method for decomposing a high-dimensional nonnegative matrixVinto the product of two nonnegative matrices,WandH, such thatV∼W H. It has been shown to have a parts-based, sparse representation of the data. NMF has been successfully applied in a variety of areas such as natural language processing, neuroscience, information retrieval, image processing, speech recognition and computational biology for the analysis and interpretation of large-scale data. There has also been simultaneous development of a related statistical latent class modeling approach, namely, probabilistic latent semantic indexing (PLSI), for analyzing and interpreting co-occurrence count data arising in natural language processing. In this paper, we present a generalized statistical approach to NMF and PLSI based on Renyi's divergence between two non-negative matrices, stemming from the Poisson likelihood. Our approach unifies various competing models and provides a unique theoretical framework for these methods. We propose a unified algorithm for NMF and provide a rigorous proof of monotonicity of multiplicative updates forWandH. In addition, we generalize the relationship between NMF and PLSI within this framework. We demonstrate the applicability and utility of our approach as well as its superior performance relative to existing methods using real-life and simulated document clustering data.
authors.affiliation: Department of Biostatistics and Bioinformatics, Fox Chase Cancer Center, Temple University Health System, Philadelphia, PA 19111, USA.
authors.forename: Karthik
authors.initials: K
authors.lastname: Devarajan
authors.affiliation: 3M Health Information Systems, Bethesda, MD 20814, USA.
authors.forename: Guoli
authors.initials: G
authors.lastname: Wang
authors.affiliation: Division of Statistics, Northern Illinois University, DeKalb, IL 60115, USA.
authors.forename: Nader
authors.initials: N
authors.lastname: Ebrahimi
keywords: Biomedical informatics; EM algorithm; Nonnegative matrix factorization; Probabilistic latent semantic indexing; Renyi's divergence; λ-log-likelihood
references.citation: PLoS Comput Biol. 2008 Jul 25;4(7):e1000029
references.pmid: 18654623
references.citation: Nature. 1999 Oct 21;401(6755):788-91
references.pmid: 10548103
references.citation: Conf Proc IEEE Eng Med Biol Soc. 2005;5:4990-3
references.pmid: 17281365
references.citation: Proc Natl Acad Sci U S A. 2004 Mar 23;101(12):4164-9
references.pmid: 15016911
references.citation: Bioinformatics. 2011 Jul 1;27(13):i401-9
references.pmid: 21685098
references.citation: Neural Comput. 2007 Oct;19(10):2756-79
references.pmid: 17716011
references.citation: Neural Comput. 2007 Mar;19(3):780-91
references.pmid: 17298233
references.citation: Neural Comput. 2012 Apr;24(4):1085-105
references.pmid: 22168561
references.citation: Bioinformatics. 2009 Feb 15;25(4):545-7
references.pmid: 19131367
references.citation: BMC Bioinformatics. 2006 Mar 28;7:175
references.pmid: 16569230
references.citation: Infect Genet Evol. 2012 Jul;12(5):913-21
references.pmid: 21930246
references.citation: BMC Bioinformatics. 2006 Jan 26;7:41
references.pmid: 16438716
references.citation: IEEE Trans Pattern Anal Mach Intell. 2006 Mar;28(3):403-15
references.pmid: 16526426