For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
Least Absolute Policy Iteration--A Robust Approach to Value Function Approximation
Masashi SUGIYAMA Hirotaka HACHIYA Hisashi KASHIMA Tetsuro MORIMURA
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2010/09/01
Online ISSN: 1745-1361
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Artificial Intelligence, Data Mining
reinforcement learning, value function approximation, least-squares policy iteration, outlier, 1-loss function, linear programming,
Full Text: PDF(2.1MB)>>
Least-squares policy iteration is a useful reinforcement learning method in robotics due to its computational efficiency. However, it tends to be sensitive to outliers in observed rewards. In this paper, we propose an alternative method that employs the absolute loss for enhancing robustness and reliability. The proposed method is formulated as a linear programming problem which can be solved efficiently by standard optimization software, so the computational advantage is not sacrificed for gaining robustness and reliability. We demonstrate the usefulness of the proposed approach through a simulated robot-control task.