title: A generalized pyramid matching kernel for human action recognition in realistic videos.
abstract: Human action recognition is an increasingly important research topic in the fields of video sensing, analysis and understanding. Caused by unconstrained sensing conditions, there exist large intra-class variations and inter-class ambiguities in realistic videos, which hinder the improvement of recognition performance for recent vision-based action recognition systems. In this paper, we propose a generalized pyramid matching kernel (GPMK) for recognizing human actions in realistic videos, based on a multi-channel "bag of words" representation constructed from local spatial-temporal features of video clips. As an extension to the spatial-temporal pyramid matching (STPM) kernel, the GPMK leverages heterogeneous visual cues in multiple feature descriptor types and spatial-temporal grid granularity levels, to build a valid similarity metric between two video clips for kernel-based classification. Instead of the predefined and fixed weights used in STPM, we present a simple, yet effective, method to compute adaptive channel weights of GPMK based on the kernel target alignment from training data. It incorporates prior knowledge and the data-driven information of different channels in a principled way. The experimental results on three challenging video datasets (i.e., Hollywood2, Youtube and HMDB51) validate the superiority of our GPMK w.r.t. the traditional STPM kernel for realistic human action recognition and outperform the state-of-the-art results in the literature.
authors.affiliation: Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai 200240, China. zhangwenjun@sjtu.edu.cn.
authors.forename: Jun
authors.initials: J
authors.lastname: Zhu
authors.forename: Quan
authors.initials: Q
authors.lastname: Zhou
authors.forename: Weijia
authors.initials: W
authors.lastname: Zou
authors.forename: Rui
authors.initials: R
authors.lastname: Zhang
authors.forename: Wenjun
authors.initials: W
authors.lastname: Zhang
mesh_terms: D000465:Algorithms; D015374:Biosensing Techniques; D006801:Humans; D010363:Pattern Recognition, Automated / Q000379:methods*; D014741:Video Recording