title: Entropy learning and relevance criteria for neural network pruning.
abstract: In this paper, entropy is a term used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence an entropy approach is proposed to dampen the early creation of such nodes by using a new computation called entropy cycle. Entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be pruned to reduce the memory requirements of the neural network.
authors.affiliation: School of Computer Engineering, Nanyang Technological University, Blk N4 2A-32, Nanyang Avenue, Singapore 639798, Singapore. asgsng@ntu.edu.sg
authors.forename: Geok See
authors.initials: GS
authors.lastname: Ng
authors.forename: Abdul
authors.initials: A
authors.lastname: Wahab
authors.forename: Daming
authors.initials: D
authors.lastname: Shi
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D019295:Computational Biology; D003198:Computer Simulation; D019277:Entropy; D005246:Feedback; D005793:Generalization, Psychological; D006801:Humans; D007858:Learning; D008568:Memory; D008959:Models, Neurological; D016571:Neural Networks, Computer; D012677:Sensation; D013663:Teaching