title: On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review.
abstract: A comprehensive review on the problem of choosing a suitable activation function for the hidden layer of a feed forward neural network has been widely investigated. Since the nonlinear component of a neural network is the main contributor to the network mapping capabilities, the different choices that may lead to enhanced performances, in terms of training, generalization, or computational costs, are analyzed, both in general-purpose and in embedded computing environments. Finally, a strategy to convert a network configuration between different activation functions without altering the network mapping capabilities will be presented.
authors.affiliation: Department of Engineering, Roma Tre University, Via Vito Volterra 62, 00146 Rome, Italy.
authors.forename: Antonino
authors.initials: A
authors.lastname: Laudani
authors.affiliation: Department of Engineering, Roma Tre University, Via Vito Volterra 62, 00146 Rome, Italy.
authors.forename: Gabriele Maria
authors.initials: GM
authors.lastname: Lozito
authors.affiliation: Department of Engineering, Roma Tre University, Via Vito Volterra 62, 00146 Rome, Italy.
authors.forename: Francesco
authors.initials: F
authors.lastname: Riganti Fulginei
authors.affiliation: Department of Engineering, Roma Tre University, Via Vito Volterra 62, 00146 Rome, Italy.
authors.forename: Alessandro
authors.initials: A
authors.lastname: Salvini
mesh_terms: D001185:Artificial Intelligence / Q000191:economics; D005246:Feedback; D006801:Humans; D016571:Neural Networks, Computer; D013663:Teaching