title: Globally convergent algorithms with local learning rates.
abstract: A novel generalized theoretical result is presented that underpins the development of globally convergent first-order batch training algorithms which employ local learning rates. This result allows us to equip algorithms of this class with a strategy for adapting the overall direction of search to a descent one. In this way, a decrease of the batch-error measure at each training iteration is ensured, and convergence of the sequence of weight iterates to a local minimizer of the batch error function is obtained from remote initial weights. The effectiveness of the theoretical result is illustrated in three application examples by comparing two well-known training algorithms with local learning rates to their globally convergent modifications.
authors.affiliation: Dept. of Inf. Syst. and Comput., Brunel Univ., London.
authors.forename: G D
authors.initials: GD
authors.lastname: Magoulas
authors.forename: V P
authors.initials: VP
authors.lastname: Plagianakos
authors.forename: M N
authors.initials: MN
authors.lastname: Vrahatis