Coupled singular value decomposition of a cross-covariance matrix.
We derive coupled on-line learning rules for the singular value decomposition (SVD) of a cross-covariance matrix. In coupled SVD rules, the singular value is estimated alongside the singular vectors, and the effective learning rates for the singular vector rules are influenced by the singular value estimates. In addition, we use a first-order approximation of Gram-Schmidt orthonormalization as decorrelation method for the estimation of multiple singular vectors and singular values. Experiments on synthetic data show that coupled learning rules converge faster than Hebbian learning rules and that the first-order approximation of Gram-Schmidt orthonormalization produces more precise estimates and better orthonormality than the standard deflation method.
Kaiser
Alexander
A
Computer Engineering Group, Faculty of Technology, Bielefeld University, D-33594 Bielefeld, Germany. akaiser@ti.uni-bielefeld.de
Schenck
Wolfram
W
Möller
Ralf
R
D000465:Algorithms; D007858:Learning; D008433:Mathematics; D025341:Principal Component Analysis; D013269:Stochastic Processes