title: Multiplex visibility graphs to investigate recurrent neural network dynamics.
abstract: A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.
authors.affiliation: Machine Learning Group, Department of Physics and Technology, University of Tromsø, 9019 Tromsø, Norway.
authors.forename: Filippo Maria
authors.initials: FM
authors.lastname: Bianchi
authors.affiliation: Department of Computer Science, College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4QF, United Kingdom.
authors.forename: Lorenzo
authors.initials: L
authors.lastname: Livi
authors.affiliation: Department of Electronics, Information, and Bioengineering, Politecnico di Milano, 20133 Milan, Italy.|Faculty of Informatics, Universitá della Svizzera Italiana, 6900 Lugano, Switzerland.
authors.forename: Cesare
authors.initials: C
authors.lastname: Alippi
authors.affiliation: Machine Learning Group, Department of Physics and Technology, University of Tromsø, 9019 Tromsø, Norway.
authors.forename: Robert
authors.initials: R
authors.lastname: Jenssen
mesh_terms: D016571:Neural Networks, Computer; D013597:Systems Analysis; D000069558:Unsupervised Machine Learning