title: Classification of complex information: inference of co-occurring affective states from their expressions in speech.
abstract: We present a classification algorithm for inferring affective states (emotions, mental states, attitudes, and the like) from their nonverbal expressions in speech. It is based on the observations that affective states can occur simultaneously and different sets of vocal features, such as intonation and speech rate, distinguish between nonverbal expressions of different affective states. The input to the inference system was a large set of vocal features and metrics that were extracted from each utterance. The classification algorithm conducted independent pairwise comparisons between nine affective-state groups. The classifier used various subsets of metrics of the vocal features and various classification algorithms for different pairs of affective-state groups. Average classification accuracy of the 36 pairwise machines was 75 percent, using 10-fold cross validation. The comparison results were consolidated into a single ranked list of the nine affective-state groups. This list was the output of the system and represented the inferred combination of co-occurring affective states for the analyzed utterance. The inference accuracy of the combined machine was 83 percent. The system automatically characterized over 500 affective state concepts from the Mind Reading database. The inference of co-occurring affective states was validated by comparing the inferred combinations to the lexical definitions of the labels of the analyzed sentences. The distinguishing capabilities of the system were comparable to human performance.
authors.affiliation: Department of Industrial Engineering andManagement, Ben-Gurion University of the Negev, POB 653, Beer-Sheva 84105, Israel. stal@bgu.ac.il
authors.forename: Tal
authors.initials: T
authors.lastname: Sobol-Shikler
authors.forename: Peter
authors.initials: P
authors.lastname: Robinson
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D020817:Asperger Syndrome / Q000503:physiopathology / Q000523:psychology; D001290:Attitude; D016208:Databases, Factual; D019260:Expressed Emotion; D006801:Humans; D009633:Nonverbal Communication; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D013060:Speech