title: Solving the linear interval tolerance problem for weight initialization of neural networks.
abstract: Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed linear interval tolerance approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods.
authors.affiliation: Computational Intelligence Laboratory, Department of Mathematics, University of Patras, GR-26110 Patras, Greece; Department of Computer Engineering, Technological Educational Institute of Epirus, 47100 Arta, Greece. Electronic address: adamsp@upatras.gr.
authors.forename: S P
authors.initials: SP
authors.lastname: Adam
authors.affiliation: Department of Automation, Technological Educational Institute of Sterea Hellas, 34400 Psahna, Evia, Greece.
authors.forename: D A
authors.initials: DA
authors.lastname: Karras
authors.affiliation: Department of Computer Science and Information Systems, Birkbeck College, University of London, Malet Street, London WC1E 7HX, UK.
authors.forename: G D
authors.initials: GD
authors.lastname: Magoulas
authors.affiliation: Computational Intelligence Laboratory, Department of Mathematics, University of Patras, GR-26110 Patras, Greece.
authors.forename: M N
authors.initials: MN
authors.lastname: Vrahatis
mesh_terms: D000465:Algorithms; D006801:Humans; D016014:Linear Models; D008962:Models, Theoretical; D016571:Neural Networks, Computer; D011340:Problem Solving; D012044:Regression Analysis
keywords: Interval analysis; Linear interval tolerance problem; Neural networks; Weight initialization