A Novel Hybrid Network Model Based on Attentional Multi-Feature Fusion for Deception Detection

Yuanbo FANG
Hongliang FU
Huawei TAO
Ruiyu LIANG
Li ZHAO

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E104-A    No.3    pp.622-626
Publication Date: 2021/03/01
Publicized: 2020/09/24
Online ISSN: 1745-1337
DOI: 10.1587/transfun.2020EAL2051
Type of Manuscript: LETTER
Category: Speech and Hearing
Keyword: 
deception detection,  hybrid network,  attention mechanism,  feature fusion,  

Full Text: PDF>>
Buy this Article



Summary: 
Speech based deception detection using deep learning is one of the technologies to realize a deception detection system with high recognition rate in the future. Multi-network feature extraction technology can effectively improve the recognition performance of the system, but due to the limited labeled data and the lack of effective feature fusion methods, the performance of the network is limited. Based on this, a novel hybrid network model based on attentional multi-feature fusion (HN-AMFF) is proposed. Firstly, the static features of large amounts of unlabeled speech data are input into DAE for unsupervised training. Secondly, the frame-level features and static features of a small amount of labeled speech data are simultaneously input into the LSTM network and the encoded output part of DAE for joint supervised training. Finally, a feature fusion algorithm based on attention mechanism is proposed, which can get the optimal feature set in the training process. Simulation results show that the proposed feature fusion method is significantly better than traditional feature fusion methods, and the model can achieve advanced performance with only a small amount of labeled data.