Hybrid of Reinforcement and Imitation Learning for Human-Like Agents

Rousslan F. J. DOSSA  Xinyu LIAN  Hirokazu NOMOTO  Takashi MATSUBARA  Kuniaki UEHARA  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E103-D   No.9   pp.1960-1970
Publication Date: 2020/09/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019EDP7298
Type of Manuscript: PAPER
Category: Artificial Intelligence, Data Mining
Keyword: 
autonomous driving,  game AI,  human-like behavior,  imitation learning,  reinforcement learning,  

Full Text: PDF(821.8KB)>>
Buy this Article




Summary: 
Reinforcement learning methods achieve performance superior to humans in a wide range of complex tasks and uncertain environments. However, high performance is not the sole metric for practical use such as in a game AI or autonomous driving. A highly efficient agent performs greedily and selfishly, and is thus inconvenient for surrounding users, hence a demand for human-like agents. Imitation learning reproduces the behavior of a human expert and builds a human-like agent. However, its performance is limited to the expert's. In this study, we propose a training scheme to construct a human-like and efficient agent via mixing reinforcement and imitation learning for discrete and continuous action space problems. The proposed hybrid agent achieves a higher performance than a strict imitation learning agent and exhibits more human-like behavior, which is measured via a human sensitivity test.