Model-Based Reinforcement Learning in Multiagent Systems with Sequential Action Selection

Ali AKRAMIZADEH  Ahmad AFSHAR  Mohammad Bagher MENHAJ  Samira JAFARI  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E94-D   No.2   pp.255-263
Publication Date: 2011/02/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E94.D.255
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Fundamentals of Information Systems
Keyword: 
multiagent systems,  Markov games,  model-based reinforcement learning,  extensive form game,  

Full Text: PDF>>
Buy this Article




Summary: 
Model-based reinforcement learning uses the gathered information, during each experience, more efficiently than model-free reinforcement learning. This is especially interesting in multiagent systems, since a large number of experiences are necessary to achieve a good performance. In this paper, model-based reinforcement learning is developed for a group of self-interested agents with sequential action selection based on traditional prioritized sweeping. Every single situation of decision making in this learning process, called extensive Markov game, is modeled as n-person general-sum extensive form game with perfect information. A modified version of backward induction is proposed for action selection, which adjusts the tradeoff between selecting subgame perfect equilibrium points, as the optimal joint actions, and learning new joint actions. The algorithm is proved to be convergent and discussed based on the new results on the convergence of the traditional prioritized sweeping.