Analysis and classification of collective behavior using generative modeling and nonlinear manifold learning.
In this paper, we build a framework for the analysis and classification of collective behavior using methods from generative modeling and nonlinear manifold learning. We represent an animal group with a set of finite-sized particles and vary known features of the group structure and motion via a class of generative models to position each particle on a two-dimensional plane. Particle positions are then mapped onto training images that are processed to emphasize the features of interest and match attainable far-field videos of real animal groups. The training images serve as templates of recognizable patterns of collective behavior and are compactly represented in a low-dimensional space called embedding manifold. Two mappings from the manifold are derived: the manifold-to-image mapping serves to reconstruct new and unseen images of the group and the manifold-to-feature mapping allows frame-by-frame classification of raw video. We validate the combined framework on datasets of growing level of complexity. Specifically, we classify artificial images from the generative model, interacting self-propelled particle model, and raw overhead videos of schooling fish obtained from the literature.
Department of Mechanical and Aerospace Engineering, Polytechnic Institute of New York University, Brooklyn, NY 11201, USA. Electronic address: sb4304@nyu.edu.
Sachit
S
Butail
Erik M
EM
Bollt
Maurizio
M
Porfiri
D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence; D001522:Behavior, Animal / Q000502:physiology*; D003198:Computer Simulation; D007090:Image Interpretation, Computer-Assisted; D008954:Models, Biological; D017711:Nonlinear Dynamics; D010363:Pattern Recognition, Automated; D012399:Rotation; D014741:Video Recording; D015027:Zebrafish / Q000502:physiology*
Classification; Collective motion; Fish schooling; Generative modeling; Isomap