Automated video-based facial expression analysis of neuropsychiatric disorders.
Deficits in emotional expression are prominent in several neuropsychiatric disorders, including schizophrenia. Available clinical facial expression evaluations provide subjective and qualitative measurements, which are based on static 2D images that do not capture the temporal dynamics and subtleties of expression changes. Therefore, there is a need for automated, objective and quantitative measurements of facial expressions captured using videos. This paper presents a computational framework that creates probabilistic expression profiles for video data and can potentially help to automatically quantify emotional expression differences between patients with neuropsychiatric disorders and healthy controls. Our method automatically detects and tracks facial landmarks in videos, and then extracts geometric features to characterize facial expression changes. To analyze temporal facial expression changes, we employ probabilistic classifiers that analyze facial expressions in individual frames, and then propagate the probabilities throughout the video to capture the temporal characteristics of facial expressions. The applications of our method to healthy controls and case studies of patients with schizophrenia and Asperger's syndrome demonstrate the capability of the video-based expression analysis method in capturing subtleties of facial expression. Such results can pave the way for a video-based method for quantitative analysis of facial expressions in clinical research of disorders that cause affective deficits.
Section of Biomedical Image Analysis, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA. peng.wang@uphs.upenn.edu <peng.wang@uphs.upenn.edu>
Peng
P
Wang
Frederick
F
Barrett
Elizabeth
E
Martin
Marina
M
Milonova
Raquel E
RE
Gur
Ruben C
RC
Gur
Christian
C
Kohler
Ragini
R
Verma
D016022:Case-Control Studies; D003431:Cross-Cultural Comparison; D004644:Emotions / Q000502:physiology*; D005149:Facial Expression; D005260:Female; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D008297:Male; D001523:Mental Disorders / Q000503:physiopathology*; D010363:Pattern Recognition, Automated; D010364:Pattern Recognition, Visual / Q000502:physiology; D010775:Photic Stimulation; D015203:Reproducibility of Results; D014741:Video Recording / Q000379:methods*