Dynamic proximity of spatio-temporal sequences.
Recurrent networks can generate spatio-temporal neural sequences of very large cycles, having an apparent random behavior. Nonetheless a proximity measure between these sequences may be defined through comparison of the synaptic weight matrices that generate them. Following the dynamic neural filter (DNF) formalism we demonstrate this concept by comparing teacher and student recurrent networks of binary neurons. We show that large sequences, providing a training set well exceeding the Cover limit, allow for good determination of the synaptic matrices. Alternatively, assuming the matrices to be known, very fast determination of the biases can be achieved. Thus, a spatio-temporal sequence may be regarded as spatio-temporal encoding of the bias vector. We introduce a linear support vector machine (SVM) variant of the DNF in order to specify an optimal weight matrix. This approach allows us to deal with noise. Spatio-temporal sequences generated by different DNFs with the same number of neurons may be compared by calculating correlations of the synaptic matrices of the reconstructed DNFs. Other types of spatio-temporal sequences need the introduction of hidden neurons, and/or the use of a kernel variant of the SVM approach. The latter is being defined as a recurrent support vector network (RSVN).
School of Physics and Astronomy, Raymond and Beverly Sackler Faculty of Exact Sciences, Tel Aviv University, Tel Aviv 69978, Israel. horn@post.tau.ac.il
David
D
Horn
Gideon
G
Dror
Brigitte
B
Quenet
D000200:Action Potentials / Q000502:physiology*; D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence; D002490:Central Nervous System / Q000502:physiology; D006801:Humans; D016014:Linear Models; D009415:Nerve Net / Q000502:physiology*; D016571:Neural Networks, Computer; D009434:Neural Pathways / Q000502:physiology*; D009474:Neurons / Q000502:physiology*; D017711:Nonlinear Dynamics; D013569:Synapses / Q000502:physiology; D009435:Synaptic Transmission / Q000502:physiology*; D013997:Time Factors