Multi-Task Approach to Reinforcement Learning for Factored-State Markov Decision Problems

Jaak SIMM  Masashi SUGIYAMA  Hirotaka HACHIYA  

IEICE TRANSACTIONS on Information and Systems   Vol.E95-D   No.10   pp.2426-2437
Publication Date: 2012/10/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E95.D.2426
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Artificial Intelligence, Data Mining
reinforcement learning,  multi-task learning,  transfer learning,  factored state models,  Markov decision process,  

Full Text: PDF(508.1KB)>>
Buy this Article

Reinforcement learning (RL) is a flexible framework for learning a decision rule in an unknown environment. However, a large number of samples are often required for finding a useful decision rule. To mitigate this problem, the concept of transfer learning has been employed to utilize knowledge obtained from similar RL tasks. However, most approaches developed so far are useful only in low-dimensional settings. In this paper, we propose a novel transfer learning idea that targets problems with high-dimensional states. Our idea is to transfer knowledge between state factors (e.g., interacting objects) within a single RL task. This allows the agent to learn the system dynamics of the target RL task with fewer data samples. The effectiveness of the proposed method is demonstrated through experiments.