title: LMS learning algorithms: misconceptions and new results on converence.
abstract: The Widrow-Hoff delta rule is one of the most popular rules used in training neural networks. It was originally proposed for the ADALINE, but has been successfully applied to a few nonlinear neural networks as well. Despite its popularity, there exist a few misconceptions on its convergence properties. In this paper we consider repetitive learning (i.e., a fixed set of samples are used for training) and provide an in-depth analysis in the least mean square (LMS) framework. Our main result is that contrary to common belief, the nonbatch Widrow-Hoff rule does not converge in general. It converges only to a limit cycle.
authors.affiliation: FAS Technologies, Dallas, TX 75238, USA. zwang@fas.com
authors.forename: Z Q
authors.initials: ZQ
authors.lastname: Wang
authors.forename: M T
authors.initials: MT
authors.lastname: Manry
authors.forename: J L
authors.initials: JL
authors.lastname: Schiano