Man-Machine Interaction Using a Vision System with Dual Viewing Angles

Ying-Jieh HUANG  Hiroshi DOHI  Mitsuru ISHIZUKA  

IEICE TRANSACTIONS on Information and Systems   Vol.E80-D   No.11   pp.1074-1083
Publication Date: 1997/11/25
Online ISSN: 
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Image Processing,Computer Graphics and Pattern Recognition
vision system,  dual viewing angles,  speech dialogue system,  motion tracking,  mouth pattern recognition,  

Full Text: PDF(1007.3KB)>>
Buy this Article

This paper describes a vision system with dual viewing angles, i. e., wide and narrow viewing angles, and a scheme of user-friendly speech dialogue environment based on the vision system. The wide viewing angle provides a wide viewing field for wide range motion tracking, and the narrow viewing angle is capable of following a target in wide viewing field to take the image of the target with sufficient resolution. For a fast and robust motion tracking, modified motion energy (MME) and existence energy (EE) are defined to detect the motion of the target and extract the motion region at the same time. Instead of using a physical device such as a foot switch commonly used in speech dialogue systems, the begin/end of an utterance is detected from the movement of user's mouth in our system. Without recognizing the movement of lips directly, the shape variation of the region between lips is tracked for more stable recognition of the span of a dialogue. The tracking speed is about 10 frames/sec when no recognition is performed and about 5 frames/sec when both tracking and recognition are performed without using any special hardware.