Optimal Digital Control with Uncertain Network Delay of Linear Systems Using Reinforcement Learning

Taishi FUJITA  Toshimitsu USHIO  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E99-A   No.2   pp.454-461
Publication Date: 2016/02/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E99.A.454
Type of Manuscript: Special Section PAPER (Special Section on Mathematical Systems Science and its Applications)
Category: 
Keyword: 
reinforcement learning,  adaptive control,  output feedback control,  optimal control,  linear system,  

Full Text: PDF(888KB)
>>Buy this Article


Summary: 
Recent development in network technology can realize the control of a remote plant by a digital controller. However, there is a delay caused by data transmission of control inputs and outputs. The delay degrades the control performance without taking it into consideration. In general, it is a difficult problem to identify the delay beforehand. We also assume that the plant's parameters have uncertainty. To solve the problem, we use reinforcement learning to achieve optimal digital control. First, we consider state feedback control. Next, we consider the case where the plant's outputs are observed, and apply reinforcement learning to output feedback control. Finally, we demonstrate by simulation that the proposed control method can search for the optimal gain and that it can adapt to the change of the delay.