title: Reduced HyperBF networks: regularization by explicit complexity reduction and scaled Rprop-based training.
abstract: Hyper basis function (HyperBF) networks are generalized radial basis function neural networks (where the activation function is a radial function of a weighted distance). Such generalization provides HyperBF networks with high capacity to learn complex functions, which in turn make them susceptible to overfitting and poor generalization. Moreover, training a HyperBF network demands the weights, centers, and local scaling factors to be optimized simultaneously. In the case of a relatively large dataset with a large network structure, such optimization becomes computationally challenging. In this paper, a new regularization method that performs soft local dimension reduction in addition to weight decay is proposed. The regularized HyperBF network is shown to provide classification accuracy competitive to a support vector machine while requiring a significantly smaller network structure. Furthermore, a practical training to construct HyperBF networks is presented. Hierarchal clustering is used to initialize neurons followed by a gradient optimization using a scaled version of the Rprop algorithm with a localized partial backtracking step. Experimental results on seven datasets show that the proposed training provides faster and smoother convergence than the regular Rprop algorithm.
authors.affiliation: Department of Genetic Medicine, Weill Cornell, Medical College, New York, NY 10065, USA. raminmahdi@yahoo.com
authors.forename: Rami N
authors.initials: RN
authors.lastname: Mahdi
authors.forename: Eric Christian
authors.initials: EC
authors.lastname: Rouchka
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation / Q000592:standards*; D001330:Electronic Data Processing / Q000379:methods / Q000592:standards; D006801:Humans; D055641:Mathematical Concepts; D016571:Neural Networks, Computer; D016011:Normal Distribution; D010363:Pattern Recognition, Automated / Q000379:methods* / Q000592:standards*; D012985:Software Design