A Real-Time Subtask-Assistance Strategy for Adaptive Services Composition

Li QUAN  Zhi-liang WANG  Xin LIU  

IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.5   pp.1361-1369
Publication Date: 2018/05/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017EDP7131
Type of Manuscript: PAPER
Category: Data Engineering, Web Information Systems
service composition,  reinforcement learning,  Q-learning,  subtask-assistance,  

Full Text: PDF(1.3MB)
>>Buy this Article

Reinforcement learning has been used to adaptive service composition. However, traditional algorithms are not suitable for large-scale service composition. Based on Q-Learning algorithm, a multi-task oriented algorithm named multi-Q learning is proposed to realize subtask-assistance strategy for large-scale and adaptive service composition. Differ from previous studies that focus on one task, we take the relationship between multiple service composition tasks into account. We decompose complex service composition task into multiple subtasks according to the graph theory. Different tasks with the same subtasks can assist each other to improve their learning speed. The results of experiments show that our algorithm could obtain faster learning speed obviously than traditional Q-learning algorithm. Compared with multi-agent Q-learning, our algorithm also has faster convergence speed. Moreover, for all involved service composition tasks that have the same subtasks between each other, our algorithm can improve their speed of learning optimal policy simultaneously in real-time.