A Technique for Estimating Intensity of Emotional Expressions and Speaking Styles in Speech Based on Multiple-Regression HSMM

Takashi NOSE  Takao KOBAYASHI  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E93-D   No.1   pp.116-124
Publication Date: 2010/01/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E93.D.116
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Speech and Hearing
Keyword: 
emotion recognition,  emotional expression,  speaking style,  intensity of style,  hidden semi-Markov model (HSMM),  multiple-regression HSMM (MRHSMM),  

Full Text: PDF(466.4KB)>>
Buy this Article




Summary: 
In this paper, we propose a technique for estimating the degree or intensity of emotional expressions and speaking styles appearing in speech. The key idea is based on a style control technique for speech synthesis using a multiple regression hidden semi-Markov model (MRHSMM), and the proposed technique can be viewed as the inverse of the style control. In the proposed technique, the acoustic features of spectrum, power, fundamental frequency, and duration are simultaneously modeled using the MRHSMM. We derive an algorithm for estimating explanatory variables of the MRHSMM, each of which represents the degree or intensity of emotional expressions and speaking styles appearing in acoustic features of speech, based on a maximum likelihood criterion. We show experimental results to demonstrate the ability of the proposed technique using two types of speech data, simulated emotional speech and spontaneous speech with different speaking styles. It is found that the estimated values have correlation with human perception.