title: Discriminative latent models for recognizing contextual group activities.
abstract: In this paper, we go beyond recognizing the actions of individuals and focus on group activities. This is motivated from the observation that human actions are rarely performed in isolation; the contextual information of what other people in the scene are doing provides a useful cue for understanding high-level activities. We propose a novel framework for recognizing group activities which jointly captures the group activity, the individual person actions, and the interactions among them. Two types of contextual information, group-person interaction and person-person interaction, are explored in a latent variable framework. In particular, we propose three different approaches to model the person-person interaction. One approach is to explore the structures of person-person interaction. Differently from most of the previous latent structured models, which assume a predefined structure for the hidden layer, e.g., a tree structure, we treat the structure of the hidden layer as a latent variable and implicitly infer it during learning and inference. The second approach explores person-person interaction in the feature level. We introduce a new feature representation called the action context (AC) descriptor. The AC descriptor encodes information about not only the action of an individual person in the video, but also the behavior of other people nearby. The third approach combines the above two. Our experimental results demonstrate the benefit of using contextual information for disambiguating group activities.
authors.affiliation: School of Computing Science, Simon Fraser University, Burnaby, BC, Canada. tla58@sfu.ca
authors.forename: Tian
authors.initials: T
authors.lastname: Lan
authors.forename: Yang
authors.initials: Y
authors.lastname: Wang
authors.forename: Weilong
authors.initials: W
authors.lastname: Yang
authors.forename: Stephen N
authors.initials: SN
authors.lastname: Robinovitch
authors.forename: Greg
authors.initials: G
authors.lastname: Mori
mesh_terms: D000058:Accidental Falls; D000203:Activities of Daily Living / Q000145:classification*; D000465:Algorithms; D001185:Artificial Intelligence; D016002:Discriminant Analysis; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D007398:Interpersonal Relations; D008962:Models, Theoretical; D009735:Nursing Homes; D012372:ROC Curve; D012919:Social Behavior; D013037:Spatial Behavior; D014741:Video Recording