A fast and convergent stochastic MLP learning algorithm.
We propose a stochastic learning algorithm for multilayer perceptrons of linear-threshold function units, which theoretically converges with probability one and experimentally exhibits 100% convergence rate and remarkable speed on parity and classification problems with typical generalization accuracy. For learning the n bit parity function with n hidden units, the algorithm converged on all the trials we tested (n=2 to 12) after 5.8 x 4.1(n) presentations for 0.23 x 4.0(n-6) seconds on a 533MHz Alpha 21164A chip on average, which is five to ten times faster than Levenberg-Marquardt algorithm with restarts. For a medium size classification problem known as Thyroid in UCI repository, the algorithm is faster in speed and comparative in generalization accuracy than the standard backpropagation and Levenberg-Marquardt algorithms.
Sakurai
A
A
Graduate School of Knowledge Science, Japan Advanced Institute of Science and Technology, Tatsunokuchi, Ishikawa923-1292, Japan. ASakurai@jaist.ac.jp
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D007858:Learning; D016014:Linear Models; D008959:Models, Neurological; D016571:Neural Networks, Computer; D012815:Signal Processing, Computer-Assisted; D013269:Stochastic Processes; D013995:Time