title: Weakly supervised learning of interactions between humans and objects.
abstract: We introduce a weakly supervised approach for learning human actions modeled as interactions between humans and objects. Our approach is human-centric: We first localize a human in the image and then determine the object relevant for the action and its spatial relation with the human. The model is learned automatically from a set of still images annotated only with the action label. Our approach relies on a human detector to initialize the model learning. For robustness to various degrees of visibility, we build a detector that learns to combine a set of existing part detectors. Starting from humans detected in a set of images depicting the action, our approach determines the action object and its spatial relation to the human. Its final output is a probabilistic model of the human-object interaction, i.e., the spatial relation between the human and the object. We present an extensive experimental evaluation on the sports action data set from [1], the PASCAL Action 2010 data set [2], and a new human-object interaction data set.
authors.affiliation: ETH Zurich, Sternwartstrasse 7, Zurich CH-8092, Switzerland. prest@vision.ee.ethz.ch
authors.forename: Alessandro
authors.initials: A
authors.lastname: Prest
authors.forename: Cordelia
authors.initials: C
authors.lastname: Schmid
authors.forename: Vittorio
authors.initials: V
authors.lastname: Ferrari
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods; D015233:Models, Statistical; D010363:Pattern Recognition, Automated / Q000379:methods*