On objective function, regularizer, and prediction error of a learning algorithm for dealing with multiplicative weight noise.
In this paper, an objective function for training a functional link network to tolerate multiplicative weight noise is presented. Basically, the objective function is similar in form to other regularizer-based functions that consist of a mean square training error term and a regularizer term. Our study shows that under some mild conditions the derived regularizer is essentially the same as a weight decay regularizer. This explains why applying weight decay can also improve the fault-tolerant ability of a radial basis function (RBF) with multiplicative weight noise. In accordance with the objective function, a simple learning algorithm for a functional link network with multiplicative weight noise is derived. Finally, the mean prediction error of the trained network is analyzed. Simulated experiments on two artificial data sets and a real-world application are performed to verify theoretical result.
Sum
John Pui-Fai
JP
Department of Electronic Engineering, City University of Hong Kong, Hong Kong. pfsum@dragon.nchu.edu.tw
Leung
Chi-Sing
CS
Ho
Kevin I-J
KI
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D016208:Databases, Factual; D016571:Neural Networks, Computer; D017711:Nonlinear Dynamics; D016011:Normal Distribution; D055609:Stars, Celestial; D013997:Time Factors