title: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences.
abstract: Automatic analysis of human facial expression is a challenging problem with many applications. Most of the existing automated systems for facial expression analysis attempt to recognize a few prototypic emotional expressions, such as anger and happiness. Instead of representing another approach to machine analysis of prototypic facial expressions of emotion, the method presented in this paper attempts to handle a large range of human facial behavior by recognizing facial muscle actions that produce expressions. Virtually all of the existing vision systems for facial muscle action detection deal only with frontal-view face images and cannot handle temporal dynamics of facial actions. In this paper, we present a system for automatic recognition of facial action units (AUs) and their temporal models from long, profile-view face image sequences. We exploit particle filtering to track 15 facial points in an input face-profile sequence, and we introduce facial-action-dynamics recognition from continuous video input using temporal rules. The algorithm performs both automatic segmentation of an input video into facial expressions pictured and recognition of temporal segments (i.e., onset, apex, offset) of 27 AUs occurring alone or in a combination in the input face-profile video. A recognition rate of 87% is achieved.
authors.affiliation: Delft University of Technology, Electrical Engineering, Mathematics and Computer Science, The Netherlands. mpantic@ieee.org
authors.forename: Maja
authors.initials: M
authors.lastname: Pantic
authors.forename: Ioannis
authors.initials: I
authors.lastname: Patras
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D016000:Cluster Analysis; D005145:Face / Q000033:anatomy & histology* / Q000502:physiology*; D005149:Facial Expression; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016247:Information Storage and Retrieval / Q000379:methods; D009068:Movement / Q000502:physiology*; D010363:Pattern Recognition, Automated / Q000379:methods*; D010781:Photography / Q000379:methods; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013382:Subtraction Technique; D013997:Time Factors; D014741:Video Recording / Q000379:methods