Nonverbal Social Withdrawal in Depression: Evidence from manual and automatic analysis.
The relationship between nonverbal behavior and severity of depression was investigated by following depressed participants over the course of treatment and video recording a series of clinical interviews. Facial expressions and head pose were analyzed from video using manual and automatic systems. Both systems were highly consistent for FACS action units (AUs) and showed similar effects for change over time in depression severity. When symptom severity was high, participants made fewer affiliative facial expressions (AUs 12 and 15) and more non-affiliative facial expressions (AU 14). Participants also exhibited diminished head motion (i.e., amplitude and velocity) when symptom severity was high. These results are consistent with the Social Withdrawal hypothesis: that depressed individuals use nonverbal behavior to maintain or increase interpersonal distance. As individuals recover, they send more signals indicating a willingness to affiliate. The finding that automatic facial expression analysis was both consistent with manual coding and revealed the same pattern of findings suggests that automatic facial expression analysis may be ready to relieve the burden of manual coding in behavioral and clinical science.
4322 Sennott Square, Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA 15260.
Jeffrey M
JM
Girard
4322 Sennott Square, Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA 15260 ; 5000 Forbes Avenue, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, 15213.
Jeffrey F
JF
Cohn
2390 S. York Street, Department of Electrical and Computer Engineering, University of Denver, Denver, CO, USA, 80208.
Mohammad H
MH
Mahoor
2390 S. York Street, Department of Electrical and Computer Engineering, University of Denver, Denver, CO, USA, 80208.
S Mohammad
SM
Mavadati
5000 Forbes Avenue, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, 15213.
Zakia
Z
Hammal
4322 Sennott Square, Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA 15260.
Dean P
DP
Rosenwald
Depression; FACS; Facial Expression; Head Motion; Multimodal