A Novel Supervised Bimodal Emotion Recognition Approach Based on Facial Expression and Body Gesture

Jingjie YAN  Guanming LU  Xiaodong BAI  Haibo LI  Ning SUN  Ruiyu LIANG  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E101-A   No.11   pp.2003-2006
Publication Date: 2018/11/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E101.A.2003
Type of Manuscript: LETTER
Category: Image
Keyword: 
facial expression,  body gesture,  bimodal emotion recognition,  feature fusion,  supervised multiset canonical correlation analysis (SMCCA),  

Full Text: PDF(283KB)
>>Buy this Article


Summary: 
In this letter, we propose a supervised bimodal emotion recognition approach based on two important human emotion modalities including facial expression and body gesture. A effectively supervised feature fusion algorithms named supervised multiset canonical correlation analysis (SMCCA) is presented to established the linear connection between three sets of matrices, which contain the feature matrix of two modalities and their concurrent category matrix. The test results in the bimodal emotion recognition of the FABO database show that the SMCCA algorithm can get better or considerable efficiency than unsupervised feature fusion algorithm covering canonical correlation analysis (CCA), sparse canonical correlation analysis (SCCA), multiset canonical correlation analysis (MCCA) and so on.