A Nonlinear Approach to Robust Routing Based on Reinforcement Learning with State Space Compression and Adaptive Basis Construction

Hideki SATOH  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E91-A   No.7   pp.1733-1740
Publication Date: 2008/07/01
Online ISSN: 1745-1337
DOI: 10.1093/ietfec/e91-a.7.1733
Print ISSN: 0916-8508
Type of Manuscript: PAPER
Category: Nonlinear Problems
Keyword: 
robust routing,  reinforcement learning,  multivariate analysis,  function approximation,  

Full Text: PDF>>
Buy this Article




Summary: 
A robust routing algorithm was developed based on reinforcement learning that uses (1) reward-weighted principal component analysis, which compresses the state space of a network with a large number of nodes and eliminates the adverse effects of various types of attacks or disturbance noises, (2) activity-oriented index allocation, which adaptively constructs a basis that is used for approximating routing probabilities, and (3) newly developed space compression based on a potential model that reduces the space for routing probabilities. This algorithm takes all the network states into account and reduces the adverse effects of disturbance noises. The algorithm thus works well, and the frequencies of causing routing loops and falling to a local optimum are reduced even if the routing information is disturbed.