Peeling or Cutting Action and Its Target Food Product Recognition Based on Relationship between the Action and the Food Product

Yoko YAMAKATA  Hideto OBARA  Atsushi SAWADA  Koh KAKUSHO  Michihiko MINOH  

Publication
D - Abstracts of IEICE TRANSACTIONS on Information and Systems (Japanese Edition)   Vol.J90-D   No.9   pp.2550-2561
Publication Date: 2007/09/01
Online ISSN: 1881-0225
DOI: 
Print ISSN: 1880-4535
Type of Manuscript: PAPER
Category: 
Keyword: 
cooking instruction content,  video indexing,  recipe,  object recognition,  action recognition,  

Full Text(in Japanese): PDF(773.2KB)
>>Buy this Article


Summary: 
In this paper, an action and a food product recognition method is proposed for the purpose of indexing cooking video that is captured a peeling or cutting action based on a recipe. A cooking action and its target food product have the following two relationships. (i) The feature change of the food product by the action depends on the interaction of the action and the food product. (ii) Possible interactions of the recognition results of action and food product are limited to the interactions that appear in the recipe. And these relationships are introduced to the method as constraint conditions. The result of the analysis of 100 recipes shows that any combination of the ten most frequently used food products, excluding two, can be distinguished by comparing color before and after cooking action. Moreover, by the relation (ii), the average number of the candidates of food product recognition decreases from 4.6 to 2.6 if the system can recognize the cooking action as one of four categories, and the number drops to 1.8 if the system can know the food's color before and after cooking action in addition to. Finally, the system has achieved 100% accuracy for both of a cooking action and its target object recognition according to a video recording that captured three popular recipes completed by an individual.