Joint Multi-Patch and Multi-Task CNNs for Robust Face Recognition

Yanfei LIU
Junhua CHEN

IEICE TRANSACTIONS on Information and Systems   Vol.E103-D    No.10    pp.2178-2187
Publication Date: 2020/10/01
Publicized: 2020/07/02
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2020EDP7059
Type of Manuscript: PAPER
Category: Pattern Recognition
CNN,  multi-task learning,  face representation learning,  face recognition,  

Full Text: PDF>>
Buy this Article

In this paper, we present a joint multi-patch and multi-task convolutional neural networks (JMM-CNNs) framework to learn more descriptive and robust face representation for face recognition. In the proposed JMM-CNNs, a set of multi-patch CNNs and a feature fusion network are constructed to learn and fuse global and local facial features, then a multi-task learning algorithm, including face recognition task and pose estimation task, is operated on the fused feature to obtain a pose-invariant face representation for the face recognition task. To further enhance the pose insensitiveness of the learned face representation, we also introduce a similarity regularization term on features of the two tasks to propose a regularization loss. Moreover, a simple but effective patch sampling strategy is applied to make the JMM-CNNs have an end-to-end network architecture. Experiments on Multi-PIE dataset demonstrate the effectiveness of the proposed method, and we achieve a competitive performance compared with state-of-the-art methods on Labeled Face in the Wild (LFW), YouTube Faces (YTF) and MegaFace Challenge.