Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning.
A lack of automated, quantitative, and accurate assessment of social behaviors in mammalian animal models has limited progress toward understanding mechanisms underlying social interactions and their disorders such as autism. Here we present a new integrated hardware and software system that combines video tracking, depth sensing, and machine learning for automatic detection and quantification of social behaviors involving close and dynamic interactions between two mice of different coat colors in their home cage. We designed a hardware setup that integrates traditional video cameras with a depth camera, developed computer vision tools to extract the body "pose" of individual animals in a social context, and used a supervised learning algorithm to classify several well-described social behaviors. We validated the robustness of the automated classifiers in various experimental settings and used them to examine how genetic background, such as that of Black and Tan Brachyury (BTBR) mice (a previously reported autism model), influences social behavior. Our integrated approach allows for rapid, automated measurement of social behaviors across diverse experimental designs and also affords the ability to develop new, objective behavioral metrics.
Division of Biology and Biological Engineering 156-29, Howard Hughes Medical Institute, California Institute of Technology, Pasadena, CA 91125; whong@caltech.edu perona@caltech.edu wuwei@caltech.edu.
Weizhe
W
Hong
Division of Biology and Biological Engineering 156-29, Howard Hughes Medical Institute, California Institute of Technology, Pasadena, CA 91125;
Ann
A
Kennedy
Division of Engineering and Applied Sciences 136-93, California Institute of Technology, Pasadena, CA 91125.
Xavier P
XP
Burgos-Artizzu
Division of Biology and Biological Engineering 156-29, Howard Hughes Medical Institute, California Institute of Technology, Pasadena, CA 91125;
Moriel
M
Zelikowsky
Division of Engineering and Applied Sciences 136-93, California Institute of Technology, Pasadena, CA 91125.
Santiago G
SG
Navonne
Division of Engineering and Applied Sciences 136-93, California Institute of Technology, Pasadena, CA 91125 whong@caltech.edu perona@caltech.edu wuwei@caltech.edu.
Pietro
P
Perona
Division of Biology and Biological Engineering 156-29, Howard Hughes Medical Institute, California Institute of Technology, Pasadena, CA 91125; whong@caltech.edu perona@caltech.edu wuwei@caltech.edu.
David J
DJ
Anderson
D000465:Algorithms; D000818:Animals; D001522:Behavior, Animal; D003201:Computers; D005260:Female; D007091:Image Processing, Computer-Assisted; D000069550:Machine Learning; D008297:Male; D051379:Mice; D008807:Mice, Inbred BALB C; D008810:Mice, Inbred C57BL; D023421:Models, Animal; D015588:Observer Variation; D010363:Pattern Recognition, Automated; D015203:Reproducibility of Results; D012919:Social Behavior; D012984:Software; D014741:Video Recording
behavioral tracking; depth sensing; machine vision; social behavior; supervised machine learning