A Framework of Real Time Hand Gesture Vision Based Human-Computer Interaction

Liang SHA  Guijin WANG  Xinggang LIN  Kongqiao WANG  

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E94-A   No.3   pp.979-989
Publication Date: 2011/03/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E94.A.979
Print ISSN: 0916-8508
Type of Manuscript: PAPER
Category: Vision
vision based human computer interaction,  hand gesture recognition,  hand gesture detection,  discriminative kernel based tracking,  histogram of oriented gradients,  

Full Text: PDF(1.6MB)>>
Buy this Article

This paper presents a robust framework of human-computer interaction from the hand gesture vision in the presence of realistic and challenging scenarios. To this end, several novel components are proposed. A hybrid approach is first proposed to automatically infer the beginning position of hand gestures of interest via jointly optimizing the regions given by an offline skin model trained from Gaussian mixture models and a specific hand gesture classifier trained from the Adaboost technique. To consistently track the hand in the context of using kernel based tracking, a semi-supervised feature selection strategy is further presented to choose the feature subspaces which appropriately represent the properties of offline hand skin cues and online foreground-background-classification cues. Taking the histogram of oriented gradients as the descriptor to represent hand gestures, a soft-decision approach is finally proposed for recognizing static hand gestures at the locations where severe ambiguity occurs and hidden Markov model based dynamic gestures are employed for interaction. Experiments on various real video sequences show the superior performance of the proposed components. In addition, the whole framework is applicable to real-time applications on general computing platforms.