title: Robust Full-Motion Recovery of Head by Dynamic Templates and Re-registration Techniques.
abstract: This paper presents a method to recover the full-motion (3 rotations and 3 translations) of the head from an input video using a cylindrical head model. Given an initial reference template of the head image and the corresponding head pose, the head model is created and full head motion is recovered automatically. The robustness of the approach is achieved by a combination of three techniques. First, we use the iteratively re-weighted least squares (IRLS) technique in conjunction with the image gradient to accommodate non-rigid motion and occlusion. Second, while tracking, the templates are dynamically updated to diminish the effects of self-occlusion and gradual lighting changes and to maintain accurate tracking even when the face moves out of view of the camera. Third, to minimize error accumulation inherent in the use of dynamic templates, we re-register images to a reference template whenever head pose is close to that in the template. The performance of the method, which runs in real time, was evaluated in three separate experiments using image sequences (both synthetic and real) for which ground truth head motion was known. The real sequences included pitch and yaw as large as 40° and 75°, respectively. The average recovery accuracy of the 3D rotations was about 3°. In a further test, the method was used as part of a facial expression analysis system intended for use with spontaneous facial behavior in which moderate head motion is common. Image data consisted of 1-minute of video from each of 10 subjects while engaged in a 2-person interview. The method successfully stabilized face and eye images allowing for 98% accuracy in automatic blink recognition.
authors.affiliation: Robotics Institute, Carnegie Mellon University.
authors.forename: Jing
authors.initials: J
authors.lastname: Xiao
authors.affiliation: Robotics Institute, Carnegie Mellon University.
authors.forename: Tsuyoshi
authors.initials: T
authors.lastname: Moriyama
authors.affiliation: Robotics Institute, Carnegie Mellon University.
authors.forename: Takeo
authors.initials: T
authors.lastname: Kanade
authors.affiliation: University of Pittsburgh.
authors.forename: Jeffrey F
authors.initials: JF
authors.lastname: Cohn
references.citation: Psychophysiology. 1999 Mar;36(2):253-63
references.pmid: 10194972
references.citation: IEEE Trans Pattern Anal Mach Intell. 1999 Oct;21(10):974
references.pmid: 21188284
references.citation: IEEE Trans Pattern Anal Mach Intell. 2001 Feb;23(2):97-115
references.pmid: 25210210
references.citation: J Pers Soc Psychol. 1997 Jun;72(6):1429-39
references.pmid: 9177024