Control of Discrete-Time Chaotic Systems with Policy-Based Deep Reinforcement Learning

Junya IKEMOTO  Toshimitsu USHIO  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E103-A   No.7   pp.885-892
Publication Date: 2020/07/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.2019EAP1154
Type of Manuscript: PAPER
Category: Nonlinear Problems
Keyword: 
reinforcement learning,  policy gradient,  deep reinforcement learning,  chaos control,  

Full Text: PDF>>
Buy this Article




Summary: 
The OGY method is one of control methods for a chaotic system. In the method, we have to calculate a target periodic orbit embedded in its chaotic attractor. Thus, we cannot use this method in the case where a precise mathematical model of the chaotic system cannot be identified. In this case, the delayed feedback control proposed by Pyragas is useful. However, even in the delayed feedback control, we need the mathematical model to determine a feedback gain that stabilizes the periodic orbit. Thus, we propose a reinforcement learning algorithm to the design of a controller for the chaotic system. Recently, reinforcement learning algorithms with deep neural networks have been paid much attention to. Those algorithms make it possible to control complex systems. We propose a controller design method consisting of two steps, where we determine a region including a target periodic point first, and make the controller learn an optimal control policy for its stabilization. The controller efficiently explores its control policy only in the region.