Personalized Food Image Classifier Considering Time-Dependent and Item-Dependent Food Distribution

Qing YU  Masashi ANZAWA  Sosuke AMANO  Kiyoharu AIZAWA  

IEICE TRANSACTIONS on Information and Systems   Vol.E102-D   No.11   pp.2120-2126
Publication Date: 2019/11/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019PCP0005
Type of Manuscript: Special Section PAPER (Special Section on Picture Coding and Image Media Processing)
food image recognition,  user-specific recognition,  incremental learning,  classifier adaptation,  

Full Text: FreePDF(2MB)

Since the development of food diaries could enable people to develop healthy eating habits, food image recognition is in high demand to reduce the effort in food recording. Previous studies have worked on this challenging domain with datasets having fixed numbers of samples and classes. However, in the real-world setting, it is impossible to include all of the foods in the database because the number of classes of foods is large and increases continually. In addition to that, inter-class similarity and intra-class diversity also bring difficulties to the recognition. In this paper, we solve these problems by using deep convolutional neural network features to build a personalized classifier which incrementally learns the user's data and adapts to the user's eating habit. As a result, we achieved the state-of-the-art accuracy of food image recognition by the personalization of 300 food records per user.