title: Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes.
abstract: A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Niño phenomenon studied in climate research.
authors.affiliation: School of Engineering and Science, Jacobs University Bremen gGmbH, 28759 Bremen, Germany. Electronic address: mathieu.galtier@inria.fr.
authors.forename: Mathieu N
authors.initials: MN
authors.lastname: Galtier
authors.affiliation: Institut für Meereskunde, Zentrum für Meeres- und Klimaforschung, Universität Hamburg, Hamburg, Germany; MINES ParisTech, 1, rue Claude Daunesse, F-06904 Sophia Antipolis Cedex, France.
authors.forename: Camille
authors.initials: C
authors.lastname: Marini
authors.affiliation: Laboratoire Analyse Géométrie et Applications, Université Paris XIII, France.
authors.forename: Gilles
authors.initials: G
authors.lastname: Wainrib
authors.affiliation: School of Engineering and Science, Jacobs University Bremen gGmbH, 28759 Bremen, Germany.
authors.forename: Herbert
authors.initials: H
authors.lastname: Jaeger
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D059000:El Nino-Southern Oscillation; D019277:Entropy; D016014:Linear Models; D016571:Neural Networks, Computer; D017711:Nonlinear Dynamics; D013269:Stochastic Processes; D013995:Time
keywords: Echo state networks; El Nino southern oscillation; Linear inverse modeling; Relative entropy; Stochastic processes