Co-Saliency Detection via Local Prediction and Global Refinement

Jun WANG  Lei HU  Ning LI  Chang TIAN  Zhaofeng ZHANG  Mingyong ZENG  Zhangkai LUO  Huaping GUAN  

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E102-A    No.4    pp.654-664
Publication Date: 2019/04/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E102.A.654
Type of Manuscript: PAPER
Category: Image
saliency detection,  deep saliency networks,  co-saliency refinement,  

Full Text: PDF(4.9MB)>>
Buy this Article

This paper presents a novel model in the field of image co-saliency detection. Previous works simply design low level handcrafted features or extract deep features based on image patches for co-saliency calculation, which neglect the entire object perception properties. Besides, they also neglect the problem of visual similar region's mismatching when designing co-saliency calculation model. To solve these problems, we propose a novel strategy by considering both local prediction and global refinement (LPGR). In the local prediction stage, we train a deep convolutional saliency detection network in an end-to-end manner which only use the fully convolutional layers for saliency map prediction to capture the entire object perception properties and reduce feature redundancy. In the global refinement stage, we construct a unified co-saliency refinement model by integrating global appearance similarity into a co-saliency diffusion function, realizing the propagation and optimization of local saliency values in the context of entire image group. To overcome the adverse effects of visual similar regions' mismatching, we innovatively incorporates the inter-images saliency spread constraint (ISC) term into our co-saliency calculation function. Experimental results on public datasets demonstrate consistent performance gains of the proposed model over the state-of-the-art methods.