Phoneme Set Design Based on Integrated Acoustic and Linguistic Features for Second Language Speech Recognition

Xiaoyun WANG  Tsuneo KATO  Seiichi YAMAMOTO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.4   pp.857-864
Publication Date: 2017/04/01
Online ISSN: 1745-1361
Type of Manuscript: PAPER
Category: Speech and Hearing
Keyword: 
second language (L2) speech recognition,  unified acoustic and linguistic objective function,  reduced phoneme set (RPS),  linguistic discrimination ability,  

Full Text: PDF(658.1KB)
>>Buy this Article


Summary: 
Recognition of second language (L2) speech is a challenging task even for state-of-the-art automatic speech recognition (ASR) systems, partly because pronunciation by L2 speakers is usually significantly influenced by the mother tongue of the speakers. Considering that the expressions of non-native speakers are usually simpler than those of native ones, and that second language speech usually includes mispronunciation and less fluent pronunciation, we propose a novel method that maximizes unified acoustic and linguistic objective function to derive a phoneme set for second language speech recognition. The authors verify the efficacy of the proposed method using second language speech collected with a translation game type dialogue-based computer assisted language learning (CALL) system. In this paper, the authors examine the performance based on acoustic likelihood, linguistic discrimination ability and integrated objective function for second language speech. Experiments demonstrate the validity of the phoneme set derived by the proposed method.