Pruning using parameter and neuronal metrics.
In this article, we introduce a measure of optimality for architecture selection algorithms for neural networks: the distance from the original network to the new network in a metric defined by the probability distributions of all possible networks. We derive two pruning algorithms, one based on a metric in parameter space and the other based on a metric in neuron space, which are closely related to well-known architecture selection algorithms, such as GOBS. Our framework extends the theoretically range of validity of GOBS and therefore can explain results observed in previous experiments. In addition, we give some computational improvements for these algorithms.
van de Laar
P
P
Theoretical Foundation, Foundation for Neural Networks, Department of Medical and Bio-Physics, University of Nijmegen, CPK1 213, Geert Grooteplein 21, 6525 EZ Nijmegen, The Netherlands.
Heskes
T
T
D000465:Algorithms; D003920:Diabetes Mellitus / Q000503:physiopathology; D007198:Indians, North American; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology*; D011336:Probability; D015203:Reproducibility of Results