title: Multi-Sensor Based State Prediction for Personal Mobility Vehicles.
abstract: This paper presents a study on multi-modal human emotional state detection while riding a powered wheelchair (PMV; Personal Mobility Vehicle) in an indoor labyrinth-like environment. The study reports findings on the habituation of human stress response during self-driving. In addition, the effects of "loss of controllability", change in the role of the driver to a passenger, are investigated via an autonomous driving modality. The multi-modal emotional state detector sensing framework consists of four sensing devices: electroencephalograph (EEG), heart inter-beat interval (IBI), galvanic skin response (GSR) and stressor level lever (in the case of autonomous riding). Physiological emotional state measurement characteristics are organized by time-scale, in terms of capturing slower changes (long-term) and quicker changes from moment-to-moment. Experimental results with fifteen participants regarding subjective emotional state reports and commercial software measurements validated the proposed emotional state detector. Short-term GSR and heart signal characterizations captured moment-to-moment emotional state during autonomous riding (Spearman correlation; ρ = 0.6, p < 0.001). Short-term GSR and EEG characterizations reliably captured moment-to-moment emotional state during self-driving (Classification accuracy; 69.7). Finally, long-term GSR and heart characterizations were confirmed to reliably capture slow changes during autonomous riding and also of emotional state during participant resting state. The purpose of this study and the exploration of various algorithms and sensors in a structured framework is to provide a comprehensive background for multi-modal emotional state prediction experiments and/or applications. Additional discussion regarding the feasibility and utility of the possibilities of these concepts are given.
authors.affiliation: Department of Dynamic Brain Imaging, Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.
authors.forename: Jamilah
authors.initials: J
authors.lastname: Abdur-Rahim
authors.affiliation: Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.
authors.forename: Yoichi
authors.initials: Y
authors.lastname: Morales
authors.affiliation: Department of Dynamic Brain Imaging, Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.
authors.forename: Pankaj
authors.initials: P
authors.lastname: Gupta
authors.affiliation: Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.|Multisensory Cognition and Computation Laboratory, Universal Communication Research Institute, National Institute of Information and Communications Technology, Soraku-gun, Kyoto, Japan.
authors.forename: Ichiro
authors.initials: I
authors.lastname: Umata
authors.affiliation: Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.
authors.forename: Atsushi
authors.initials: A
authors.lastname: Watanabe
authors.affiliation: Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.
authors.forename: Jani
authors.initials: J
authors.lastname: Even
authors.affiliation: Department of Dynamic Brain Imaging, Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.
authors.forename: Takayuki
authors.initials: T
authors.lastname: Suyama
authors.affiliation: Department of Dynamic Brain Imaging, Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International, Soraku-gun, Kyoto, Japan.|Department of System Science, Graduate School of Informatics, Kyoto University, Kyoto, Japan.
authors.forename: Shin
authors.initials: S
authors.lastname: Ishii
mesh_terms: D000465:Algorithms; D001921:Brain / Q000000981:diagnostic imaging; D004569:Electroencephalography; D004644:Emotions / Q000502:physiology; D005260:Female; D005712:Galvanic Skin Response / Q000502:physiology; D006185:Habituation, Psychophysiologic; D064368:Healthy Volunteers; D006339:Heart Rate / Q000502:physiology; D006801:Humans; D008297:Male; D013315:Stress, Psychological; D014910:Wheelchairs