title: Tracking people's hands and feet using mixed network AND/OR search.
abstract: We describe a framework that leverages mixed probabilistic and deterministic networks and their AND/OR search space to efficiently find and track the hands and feet of multiple interacting humans in 2D from a single camera view. Our framework detects and tracks multiple people's heads, hands, and feet through partial or full occlusion; requires few constraints (does not require multiple views, high image resolution, knowledge of performed activities, or large training sets); and makes use of constraints and AND/OR Branch-and-Bound with lazy evaluation and carefully computed bounds to efficiently solve the complex network that results from the consideration of interperson occlusion. Our main contributions are: 1) a multiperson part-based formulation that emphasizes extremities and allows for the globally optimal solution to be obtained in each frame, and 2) an efficient and exact optimization scheme that relies on AND/OR Branch-and-Bound, lazy factor evaluation, and factor cost sensitive bound computation. We demonstrate our approach on three datasets: the public single person HumanEva dataset, outdoor sequences where multiple people interact in a group meeting scenario, and outdoor one-on-one basketball videos. The first dataset demonstrates that our framework achieves state-of-the-art performance in the single person setting, while the last two demonstrate robustness in the presence of partial and full occlusion and fast nontrivial motion.
authors.affiliation: Department of Computer Science, Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742, USA. morariu@cs.umd.edu
authors.forename: Vlad I
authors.initials: VI
authors.lastname: Morariu
authors.forename: David
authors.initials: D
authors.lastname: Harwood
authors.forename: Larry S
authors.initials: LS
authors.lastname: Davis
mesh_terms: D001490:Basketball; D016208:Databases, Factual; D016002:Discriminant Analysis; D005528:Foot / Q000502:physiology*; D006225:Hand / Q000502:physiology*; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D016018:Least-Squares Analysis; D009043:Motor Activity / Q000502:physiology*; D010363:Pattern Recognition, Automated / Q000379:methods*; D014741:Video Recording