title: Inaccessibility in online learning of recurrent neural networks.
abstract: We apply nonlinear dynamical system techniques to recurrent neural networks. In particular, we numerically analyze the dynamical system characteristics of the online learning process. By introducing the notion of inaccessibility, we show that the learning process is well characterized by strong nonhyperbolicity and inaccessibility, which is a greater uncertainty than chaotic unpredictability. These results are clearly contrasted with a gradient descent dynamics, or ordinary chaos.
authors.affiliation: Future University-Hakodate, 116-2 Kameda Nakano-cho, Hakodate, Hokkaido 041-8655, Japan.
authors.forename: Asaki
authors.initials: A
authors.lastname: Saito
authors.forename: Makoto
authors.initials: M
authors.lastname: Taiji
authors.forename: Takashi
authors.initials: T
authors.lastname: Ikegami
mesh_terms: D001185:Artificial Intelligence; D007858:Learning; D008962:Models, Theoretical; D016571:Neural Networks, Computer