Hotspot Modeling of Hand-Machine Interaction Experiences from a Head-Mounted RGB-D Camera

Longfei CHEN  Yuichi NAKAMURA  Kazuaki KONDO  Walterio MAYOL-CUEVAS  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E102-D   No.2   pp.319-330
Publication Date: 2019/02/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2018EDP7146
Type of Manuscript: PAPER
Category: Human-computer Interaction
Keyword: 
egocentric vision,  machine operation experiences,  hotspots,  RGB-D,  task modeling,  

Full Text: PDF(1.7MB)
>>Buy this Article


Summary: 
This paper presents an approach to analyze and model tasks of machines being operated. The executions of the tasks were captured through egocentric vision. Each task was decomposed into a sequence of physical hand-machine interactions, which are described with touch-based hotspots and interaction patterns. Modeling the tasks was achieved by integrating the experiences of multiple experts and using a hidden Markov model (HMM). Here, we present the results of more than 70 recorded egocentric experiences of the operation of a sewing machine. Our methods show good potential for the detection of hand-machine interactions and modeling of machine operation tasks.