Speech Recognition Using HMM Based on Fusion of Visual and Auditory Information

Akira SHINTANI  Akio OGIHARA  Yoshikazu YAMAGUCHI  Yasuhisa HAYASHI  Kunio FUKUNAGA  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E77-A   No.11   pp.1875-1878
Publication Date: 1994/11/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8508
Type of Manuscript: Special Section LETTER (Special Section of Letters Selected from the 1994 IEICE Spring Conference)
Category: 
Keyword: 
speech recognition,  fusion of visual and auditory,  sensor fusion,  Hidden Markov Model,  

Full Text: PDF(239.5KB)>>
Buy this Article




Summary: 
We propose two methods to fuse auditory information and visual information for accurate sppech recognition. The first method fuses two kinds of information by using linear combination after calculating two kinds of probabilities by HMM for each word. The second method fuses two kinds of information by using the histogram which expresses the correlation of them. We have performed experiments comparing the proposed methods with the conventional method and confirmed the validity of the proposed methods.