A Method of Recipe to Cooking Video Mapping for Automated Cooking Content Construction


D - Abstracts of IEICE TRANSACTIONS on Information and Systems (Japanese Edition)   Vol.J90-D   No.10   pp.2817-2829
Publication Date: 2007/10/01
Online ISSN: 1881-0225
Print ISSN: 1880-4535
Type of Manuscript: PAPER
cooking instruction content,  video indexing,  recipe,  object recognition,  action recognition,  

Full Text(in Japanese): PDF(610KB)
>>Buy this Article

We propose a method of mapping each cooking procedure of recipe to the corresponding segment of observed multi-modal data of the cooking activity. Because the order of the cooking procedures is indicated only as partial order in the recipe, we convert the observed data and recipe to a tree-type format that expresses the order of procedure toward each food product, and map one tree to the other as node-to-node mapping. With this method, we show 72.9% of cooking action and 79.5% of food products are indexed in average when cooking actions are recognized as five categories and food products are recognized as six colors. Then, we introduce edit distance based tree mapping algorithm to deal with the recognition errors. The implemented system gives correct node-to-node mapping in actual environment.