A Hybrid Speech Emotion Recognition System Based on Spectral and Prosodic Features

Yu ZHOU  Junfeng LI  Yanqing SUN  Jianping ZHANG  Yonghong YAN  Masato AKAGI  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E93-D   No.10   pp.2813-2821
Publication Date: 2010/10/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E93.D.2813
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Human-computer Interaction
Keyword: 
speech emotion recognition,  non-uniform subband processing,  spectral feature,  prosodic feature,  

Full Text: PDF>>
Buy this Article




Summary: 
In this paper, we present a hybrid speech emotion recognition system exploiting both spectral and prosodic features in speech. For capturing the emotional information in the spectral domain, we propose a new spectral feature extraction method by applying a novel non-uniform subband processing, instead of the mel-frequency subbands used in Mel-Frequency Cepstral Coefficients (MFCC). For prosodic features, a set of features that are closely correlated with speech emotional states are selected. In the proposed hybrid emotion recognition system, due to the inherently different characteristics of these two kinds of features (e.g., data size), the newly extracted spectral features are modeled by Gaussian Mixture Model (GMM) and the selected prosodic features are modeled by Support Vector Machine (SVM). The final result of the proposed emotion recognition system is obtained by combining the results from these two subsystems. Experimental results show that (1) the proposed non-uniform spectral features are more effective than the traditional MFCC features for emotion recognition; (2) the proposed hybrid emotion recognition system using both spectral and prosodic features yields the relative recognition error reduction rate of 17.0% over the traditional recognition systems using only the spectral features, and 62.3% over those using only the prosodic features.