Semantic Motion Signature for Segmentation of High Speed Large Displacement Objects

Yinhui ZHANG  Zifen HE  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.1   pp.220-224
Publication Date: 2017/01/01
Publicized: 2016/10/05
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2016EDL8083
Type of Manuscript: LETTER
Category: Image Processing and Video Processing
Keyword: 
video object segmentation,  motion signature,  semantic prediction,  large displacement,  high speed video,  

Full Text: PDF(1MB)>>
Buy this Article




Summary: 
This paper presents a novel method for unsupervised segmentation of objects with large displacements in high speed video sequences. Our general framework introduces a new foreground object predicting method that finds object hypotheses by encoding both spatial and temporal features via a semantic motion signature scheme. More specifically, temporal cues of object hypotheses are captured by the motion signature proposed in this paper, which is derived from sparse saliency representation imposed on magnitude of optical flow field. We integrate semantic scores derived from deep networks with location priors that allows us to directly estimate appearance potentials of foreground hypotheses. A unified MRF energy functional is proposed to simultaneously incorporate the information from the motion signature and semantic prediction features. The functional enforces both spatial and temporal consistency and impose appearance constancy and spatio-temporal smoothness constraints directly on the object hypotheses. It inherently handles the challenges of segmenting ambiguous objects with large displacements in high speed videos. Our experiments on video object segmentation benchmarks demonstrate the effectiveness of the proposed method for segmenting high speed objects despite the complicated scene dynamics and large displacements.