New training strategies for constructive neural networks with application to regression problems.
Regression problem is an important application area for neural networks (NNs). Among a large number of existing NN architectures, the feedforward NN (FNN) paradigm is one of the most widely used structures. Although one-hidden-layer feedforward neural networks (OHL-FNNs) have simple structures, they possess interesting representational and learning capabilities. In this paper, we are interested particularly in incremental constructive training of OHL-FNNs. In the proposed incremental constructive training schemes for an OHL-FNN, input-side training and output-side training may be separated in order to reduce the training time. A new technique is proposed to scale the error signal during the constructive learning process to improve the input-side training efficiency and to obtain better generalization performance. Two pruning methods for removing the input-side redundant connections have also been applied. Numerical simulations demonstrate the potential and advantages of the proposed strategies when compared to other existing techniques in the literature.
Ma
L
L
Department of Electrical and Computer Engineering, Concordia University, 1455 De Maisonneuve Blvd West, Montreal, Que. H3G 1M8, Canada. liying@ece.concordia.ca
Khorasani
K
K
D000465:Algorithms; D003198:Computer Simulation; D005246:Feedback; D006801:Humans; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D011337:Probability Learning; D012044:Regression Analysis