title: Real Time 3D Facial Movement Tracking Using a Monocular Camera.
abstract: The paper proposes a robust framework for 3D facial movement tracking in real time using a monocular camera. It is designed to estimate the 3D face pose and local facial animation such as eyelid movement and mouth movement. The framework firstly utilizes the Discriminative Shape Regression method to locate the facial feature points on the 2D image and fuses the 2D data with a 3D face model using Extended Kalman Filter to yield 3D facial movement information. An alternating optimizing strategy is adopted to fit to different persons automatically. Experiments show that the proposed framework could track the 3D facial movement across various poses and illumination conditions. Given the real face scale the framework could track the eyelid with an error of 1 mm and mouth with an error of 2 mm. The tracking result is reliable for expression analysis or mental state inference.
authors.affiliation: School of Electronics and Information Engineering, Tongji University, Caoan Road 4800, Shanghai 201804, China. dongyanchao@tongji.edu.cn.
authors.forename: Yanchao
authors.initials: Y
authors.lastname: Dong
authors.affiliation: School of Electronics and Information Engineering, Tongji University, Caoan Road 4800, Shanghai 201804, China. 11wanggyanming@tongji.edu.cn.
authors.forename: Yanming
authors.initials: Y
authors.lastname: Wang
authors.affiliation: School of Electronics and Information Engineering, Tongji University, Caoan Road 4800, Shanghai 201804, China. yuejiguang@tongji.edu.cn.
authors.forename: Jiguang
authors.initials: J
authors.lastname: Yue
authors.affiliation: Department of Electrical and Electronics, Kumamoto University, 2-39-1 Kurokami, Kumamoto shi 8608555, Japan. hu@cs.kumamoto-u.ac.jp.
authors.forename: Zhencheng
authors.initials: Z
authors.lastname: Hu
mesh_terms: D000465:Algorithms; D005145:Face / Q000502:physiology*; D005149:Facial Expression; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000379:methods*; D009068:Movement / Q000502:physiology*
keywords: 3D facial movement; HCI; eyelid; facial animation; facial feature points