Tracking motion, deformation, and texture using conditionally gaussian processes.
We present a generative model and inference algorithm for 3D nonrigid object tracking. The model, which we call G-flow, enables the joint inference of 3D position, orientation, and nonrigid deformations, as well as object texture and background texture. Optimal inference under G-flow reduces to a conditionally Gaussian stochastic filtering problem. The optimal solution to this problem reveals a new space of computer vision algorithms, of which classic approaches such as optic flow and template matching are special cases that are optimal only under special circumstances. We evaluate G-flow on the problem of tracking facial expressions and head motion in 3D from single-camera video. Previously, the lack of realistic video data with ground truth nonrigid position information has hampered the rigorous evaluation of nonrigid tracking. We introduce a practical method of obtaining such ground truth data and present a new face video data set that was created using this technique. Results on this data set show that G-flow is much more robust and accurate than current deterministic optic-flow-based approaches.
Marks
Tim K
TK
Mitsubishi Electric Research Laboratories, 201 Broadway, Cambridge, MA 02139, USA. tmarks@merl.com
Hershey
John R
JR
Movellan
Javier R
JR
D000465:Algorithms; D005145:Face / Q000033:anatomy & histology*; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D009068:Movement / Q000502:physiology*; D016011:Normal Distribution; D010363:Pattern Recognition, Automated / Q000379:methods*; D013269:Stochastic Processes; D014741:Video Recording