Use of Image, Vibration Sound and Load in Cutting Action for Ingredients Recognition

Jin INOUE  Atsushi HASHIMOTO  Kazuaki NAKAMURA  Takuya FUNATOMI  Yoko YAMAKATA  Mayumi UEDA  Michihiko MINOH  

D - Abstracts of IEICE TRANSACTIONS on Information and Systems (Japanese Edition)   Vol.J97-D   No.9   pp.1490-1502
Publication Date: 2014/09/01
Online ISSN: 1881-0225
Type of Manuscript: PAPER
ingredient recognition,  multimodality,  load sensor,  auditory analysis,  cooking support,  

Full Text(in Japanese): PDF(1.7MB)
>>Buy this Article

In this paper, we propose an ingredient recognition method which makes use of a cooking behaviour, namely cutting ingredients. Typical object recognition methods use only images. However, ingredients have similar color even though they are in different categories. Shape of them, in opposition to color, is often different even in the same category. Hence, it is difficult to recognize ingredient with only images. In this paper, we involve load and vibration sound observed when cutting ingredients into our recognition frame work. Load observed on chopping board reflects hardness of ingredients, and vibration sound reflects inner structure of ingredients. These modalities contribute with image to improving recognition accuracy. We evaluated the improvement of recognition accuracy for combination of image, load and vibration sound with 23 classes of ingredients.