An Isolated Word Speech Recognition Using Fusion of Auditory and Visual Information

Akira SHINTANI
Akiko OGIHARA
Naoshi DOI
Shinobu TAKAMATSU

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E79-A    No.6    pp.777-783
Publication Date: 1996/06/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8508
Type of Manuscript: Special Section PAPER (Special Section of Papers Selected from 1995 Joint Technical Conference on Circuits/Systems, Computers and Communications (JTC-CSCC '95))
Category: 
Keyword: 
HMM,  fusion,  linear combination,  speech recognition,  auditory and visual information,  

Full Text: PDF>>
Buy this Article



Summary: 
We propose a speech recognition method using fusion of auditory and visual information for accurate speech recognition. Since we use both auditory information and visual information, we can perform speech recognition more accurately in comparison with the case of either auditory information or visual information. After processing each information by HMM, they are fused by linear combination with weight coefficient. We performed experiments and confirmed the validity of the proposed method.