title: User-state sensing for virtual health agents and telehealth applications.
abstract: Nonverbal behaviors play a crucial role in shaping outcomes in face-to-face clinical interactions. Experienced clinicians use nonverbals to foster rapport and "read" their clients to inform diagnoses. The rise of telemedicine and virtual health agents creates new opportunities, but it also strips away much of this nonverbal channel. Recent advances in low-cost computer vision and sensing technologies have the potential to address this challenge by learning to recognize nonverbal cues from large datasets of clinical interactions. These techniques can enhance both telemedicine and the emerging technology of virtual health agents. This article describes our current research in addressing these challenges in the domain of PTSD and depression screening for U.S. Veterans. We describe our general approach and report on our initial contribution: the creation of a large dataset of clinical interview data that facilitates the training of user-state sensing technology.
authors: University of Southern California, CA, USA. gratch@ict.usc.edu
authors: Jonathan
authors: J
authors: Gratch
authors: Louis-Philippe
authors: LP
authors: Morency
authors: Stefan
authors: S
authors: Scherer
authors: Giota
authors: G
authors: Stratou
authors: Jill
authors: J
authors: Boberg
authors: Sebastian
authors: S
authors: Koenig
authors: Todd
authors: T
authors: Adamson
authors: Albert
authors: A
authors: Rizzo
mesh_terms: D003936:Diagnosis, Computer-Assisted / Q000379:methods*; D005103:Expert Systems; D006801:Humans; D008487:Medical History Taking / Q000379:methods*; D001523:Mental Disorders / Q000175:diagnosis*; D009633:Nonverbal Communication; D010817:Physician-Patient Relations; D017216:Telemedicine / Q000379:methods*; D014584:User-Computer Interface