title: Echo state Gaussian process.
abstract: Echo state networks (ESNs) constitute a novel approach to recurrent neural network (RNN) training, with an RNN (the reservoir) being generated randomly, and only a readout being trained using a simple computationally efficient algorithm. ESNs have greatly facilitated the practical application of RNNs, outperforming classical approaches on a number of benchmark tasks. In this paper, we introduce a novel Bayesian approach toward ESNs, the echo state Gaussian process (ESGP). The ESGP combines the merits of ESNs and Gaussian processes to provide a more robust alternative to conventional reservoir computing networks while also offering a measure of confidence on the generated predictions (in the form of a predictive distribution). We exhibit the merits of our approach in a number of applications, considering both benchmark datasets and real-world applications, where we show that our method offers a significant enhancement in the dynamical data modeling capabilities of ESNs. Additionally, we also show that our method is orders of magnitude more computationally efficient compared to existing Gaussian process-based methods for dynamical data modeling, without compromises in the obtained predictive performance.
authors.affiliation: Department of Electrical and Electronic Engineering, Imperial College London, London, U.K. soteri0s@me.com
authors.forename: Sotirios P
authors.initials: SP
authors.lastname: Chatzis
authors.forename: Yiannis
authors.initials: Y
authors.lastname: Demiris
mesh_terms: D001499:Bayes Theorem; D006801:Humans; D007858:Learning; D016571:Neural Networks, Computer; D009474:Neurons; D017711:Nonlinear Dynamics; D016011:Normal Distribution; D010363:Pattern Recognition, Automated / Q000379:methods*; D011595:Psychomotor Agitation; D011597:Psychomotor Performance