A Novel Supervised Bimodal Emotion Recognition Approach Based on Facial Expression and Body Gesture

Jingjie YAN
Guanming LU
Xiaodong BAI
Haibo LI
Ning SUN

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E101-A    No.11    pp.2003-2006
Publication Date: 2018/11/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E101.A.2003
Type of Manuscript: LETTER
Category: Image
facial expression,  body gesture,  bimodal emotion recognition,  feature fusion,  supervised multiset canonical correlation analysis (SMCCA),  

Full Text: PDF(283KB)>>
Buy this Article

In this letter, we propose a supervised bimodal emotion recognition approach based on two important human emotion modalities including facial expression and body gesture. A effectively supervised feature fusion algorithms named supervised multiset canonical correlation analysis (SMCCA) is presented to established the linear connection between three sets of matrices, which contain the feature matrix of two modalities and their concurrent category matrix. The test results in the bimodal emotion recognition of the FABO database show that the SMCCA algorithm can get better or considerable efficiency than unsupervised feature fusion algorithm covering canonical correlation analysis (CCA), sparse canonical correlation analysis (SCCA), multiset canonical correlation analysis (MCCA) and so on.