title: Machine vision methods for analyzing social interactions.
abstract: Recent developments in machine vision methods for automatic, quantitative analysis of social behavior have immensely improved both the scale and level of resolution with which we can dissect interactions between members of the same species. In this paper, we review these methods, with a particular focus on how biologists can apply them to their own work. We discuss several components of machine vision-based analyses: methods to record high-quality video for automated analyses, video-based tracking algorithms for estimating the positions of interacting animals, and machine learning methods for recognizing patterns of interactions. These methods are extremely general in their applicability, and we review a subset of successful applications of them to biological questions in several model systems with very different types of social behaviors.
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus, 19700 Helix Drive, Ashburn, VA 20147, USA.
authors.forename: Alice A
authors.initials: AA
authors.lastname: Robie
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus, 19700 Helix Drive, Ashburn, VA 20147, USA.
authors.forename: Kelly M
authors.initials: KM
authors.lastname: Seagraves
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus, 19700 Helix Drive, Ashburn, VA 20147, USA egnorr@janelia.hhmi.org bransonk@janelia.hhmi.org.
authors.forename: S E Roian
authors.identifier: 0000-0003-0992-6240
authors.initials: SE
authors.lastname: Egnor
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus, 19700 Helix Drive, Ashburn, VA 20147, USA egnorr@janelia.hhmi.org bransonk@janelia.hhmi.org.
authors.forename: Kristin
authors.identifier: 0000-0002-5567-2512
authors.initials: K
authors.lastname: Branson
mesh_terms: D000465:Algorithms; D000818:Animals; D001522:Behavior, Animal; D004867:Equipment Design; D000069550:Machine Learning; D012919:Social Behavior; D014741:Video Recording / Q000295:instrumentation / Q000379:methods*
keywords: Animal behavior; Computer vision; Machine learning; Social behavior