Distributed Reinforcement Learning Approach for Vehicular Ad Hoc Networks

Celimuge WU  Kazuya KUMEKAWA  Toshihiko KATO 

Publication
IEICE TRANSACTIONS on Communications  Vol.E93-B  No.6  pp.1431-1442
Publication Date: 2010/06/01
Online ISSN: 1745-1345
Print ISSN: 0916-8516
Type of Manuscript: PAPER
Category: Network
Keyword: 
vehicular ad hoc networks,  routing protocol,  Q-Learning,  AODV, 

Full Text: PDF(987.9KB)

Summary: 
In Vehicular Ad hoc Networks (VANETs), general purpose ad hoc routing protocols such as AODV cannot work efficiently due to the frequent changes in network topology caused by vehicle movement. This paper proposes a VANET routing protocol QLAODV (Q-Learning AODV) which suits unicast applications in high mobility scenarios. QLAODV is a distributed reinforcement learning routing protocol, which uses a Q-Learning algorithm to infer network state information and uses unicast control packets to check the path availability in a real time manner in order to allow Q-Learning to work efficiently in a highly dynamic network environment. QLAODV is favored by its dynamic route change mechanism, which makes it capable of reacting quickly to network topology changes. We present an analysis of the performance of QLAODV by simulation using different mobility models. The simulation results show that QLAODV can efficiently handle unicast applications in VANETs.