Learning to imitate stochastic time series in a compositional way by chaos.
This study shows that a mixture of RNN experts model can acquire the ability to generate sequences that are combination of multiple primitive patterns by means of self-organizing chaos. By training the model, each expert learns a primitive sequence pattern, and a gating network learns to imitate stochastic switching of the multiple primitives via chaotic dynamics, utilizing a sensitive dependence on initial conditions. As a demonstration, we present a numerical simulation in which the model learns Markov chain switching among some Lissajous curves by chaotic dynamics. Our analysis shows that by using a sufficient amount of training data, balanced with the network memory capacity, it is possible to satisfy the conditions for embedding the target stochastic sequences into a chaotic dynamical system. It is also shown that reconstruction of a stochastic time series by a chaotic model can be stabilized by adding a negligible amount of noise to the dynamics of the model.
Namikawa
Jun
J
Brain Science Institute, RIKEN, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan. jnamika@bdc.brain.riken.go.jp
Tani
Jun
J
D000465:Algorithms; D001185:Artificial Intelligence; D001331:Automation; D003198:Computer Simulation; D005246:Feedback; D008390:Markov Chains; D008568:Memory; D016571:Neural Networks, Computer; D011336:Probability; D013269:Stochastic Processes; D013997:Time Factors