title: Neural network training as a dissipative process.
abstract: This paper analyzes the practical issues and reports some results on a theory in which learning is modeled as a continuous temporal process driven by laws describing the interactions of intelligent agents with their own environment. The classic regularization framework is paired with the idea of temporal manifolds by introducing the principle of least cognitive action, which is inspired by the related principle of mechanics. The introduction of the counterparts of the kinetic and potential energy leads to an interpretation of learning as a dissipative process. As an example, we apply the theory to supervised learning in neural networks and show that the corresponding Euler-Lagrange differential equations can be connected to the classic gradient descent algorithm on the supervised pairs. We give preliminary experiments to confirm the soundness of the theory.
authors.affiliation: Department of Information Engineering and Mathematics, University of Siena, Italy. Electronic address: marco.gori@unisi.it.
authors.forename: Marco
authors.initials: M
authors.lastname: Gori
authors.affiliation: Department of Information Engineering and Mathematics, University of Siena, Italy. Electronic address: marco.maggini@unisi.it.
authors.forename: Marco
authors.initials: M
authors.lastname: Maggini
authors.affiliation: Department of Information Engineering and Mathematics, University of Siena, Italy. Electronic address: rossi111@unisi.it.
authors.forename: Alessandro
authors.initials: A
authors.lastname: Rossi
mesh_terms: D000161:Acoustic Stimulation / Q000379:methods*; D000465:Algorithms; D006801:Humans; D000069550:Machine Learning / Q000639:trends; D008962:Models, Theoretical; D016571:Neural Networks, Computer
keywords: Dissipative systems; On-line back-propagation; Regularization networks; Temporal manifolds