Combining Multiple Acoustic Models in GMM Spaces for Robust Speech Recognition

Byung Ok KANG  Oh-Wook KWON  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E99-D   No.3   pp.724-730
Publication Date: 2016/03/01
Publicized: 2015/11/24
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2015EDP7252
Type of Manuscript: PAPER
Category: Speech and Hearing
Keyword: 
noise-robust speech recognition,  acoustic model,  GMM combination,  non-native speech recognition,  

Full Text: PDF>>
Buy this Article




Summary: 
We propose a new method to combine multiple acoustic models in Gaussian mixture model (GMM) spaces for robust speech recognition. Even though large vocabulary continuous speech recognition (LVCSR) systems are recently widespread, they often make egregious recognition errors resulting from unavoidable mismatch of speaking styles or environments between the training and real conditions. To handle this problem, a multi-style training approach has been used conventionally to train a large acoustic model by using a large speech database with various kinds of speaking styles and environment noise. But, in this work, we combine multiple sub-models trained for different speaking styles or environment noise into a large acoustic model by maximizing the log-likelihood of the sub-model states sharing the same phonetic context and position. Then the combined acoustic model is used in a new target system, which is robust to variation in speaking style and diverse environment noise. Experimental results show that the proposed method significantly outperforms the conventional methods in two tasks: Non-native English speech recognition for second-language learning systems and noise-robust point-of-interest (POI) recognition for car navigation systems.