Unconstrained Facial Expression Recognition Based on Feature Enhanced CNN and Cross-Layer LSTM

Ying TONG  Rui CHEN  Ruiyu LIANG  

IEICE TRANSACTIONS on Information and Systems   Vol.E103-D   No.11   pp.2403-2406
Publication Date: 2020/11/01
Publicized: 2020/07/30
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2020EDL8065
Type of Manuscript: LETTER
Category: Image Recognition, Computer Vision
facial expression recognition,  video sequence,  long short-term memory,  feature extraction,  

Full Text: PDF>>
Buy this Article

LSTM network have shown to outperform in facial expression recognition of video sequence. In view of limited representation ability of single-layer LSTM, a hierarchical attention model with enhanced feature branch is proposed. This new network architecture consists of traditional VGG-16-FACE with enhanced feature branch followed by a cross-layer LSTM. The VGG-16-FACE with enhanced branch extracts the spatial features as well as the cross-layer LSTM extracts the temporal relations between different frames in the video. The proposed method is evaluated on the public emotion databases in subject-independent and cross-database tasks and outperforms state-of-the-art methods.