Mining Spatial Temporal Saliency Structure for Action Recognition

Yinan LIU  Qingbo WU  Linfeng XU  Bo WU  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E99-D   No.10   pp.2643-2646
Publication Date: 2016/10/01
Publicized: 2016/07/06
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2016EDL8093
Type of Manuscript: LETTER
Category: Pattern Recognition
Keyword: 
action recognition,  video saliency,  feature pooling,  

Full Text: PDF>>
Buy this Article




Summary: 
Traditional action recognition approaches use pre-defined rigid areas to process the space-time information, e.g. spatial pyramids, cuboids. However, most action categories happen in an unconstrained manner, that is, the same action in different videos can happen at different places. Thus we need a better video representation to deal with the space-time variations. In this paper, we introduce the idea of mining spatial temporal saliency. To better handle the uniqueness of each video, we use a space-time over-segmentation approach, e.g. supervoxel. We choose three different saliency measures that take not only the appearance cues, but also the motion cues into consideration. Furthermore, we design a category-specific mining process to find the discriminative power in each action category. Experiments on action recognition datasets such as UCF11 and HMDB51 show that the proposed spatial temporal saliency video representation can match or surpass some of the state-of-the-art alternatives in the task of action recognition.