Natural Facial and Head Behavior Recognition using Dictionary of Motion Primitives

Qun SHI  Norimichi UKITA  Ming-Hsuan YANG  

IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.12   pp.2993-3000
Publication Date: 2017/12/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017EDP7128
Type of Manuscript: PAPER
Category: Image Recognition, Computer Vision
facial and head behaviors,  dynamical systems,  motion primitives,  

Full Text: PDF(1.1MB)
>>Buy this Article

This paper proposes a natural facial and head behavior recognition method using hybrid dynamical systems. Most existing facial and head behavior recognition methods focus on analyzing deliberately displayed prototypical emotion patterns rather than complex and spontaneous facial and head behaviors in natural conversation environments. We first capture spatio-temporal features on important facial parts via dense feature extraction. Next, we cluster the spatio-temporal features using hybrid dynamical systems, and construct a dictionary of motion primitives to cover all possible elemental motion dynamics accounting for facial and head behaviors. With this dictionary, the facial and head behavior can be interpreted into a distribution of motion primitives. This interpretation is robust against different rhythms of dynamic patterns in complex and spontaneous facial and head behaviors. We evaluate the proposed approach under natural tele-communication scenarios, and achieve promising results. Furthermore, the proposed method also performs favorably against the state-of-the-art methods on three benchmark databases.