Distributed Reinforcement Learning Approach for Vehicular Ad Hoc Networks

Celimuge WU  Kazuya KUMEKAWA  Toshihiko KATO  

IEICE TRANSACTIONS on Communications   Vol.E93-B   No.6   pp.1431-1442
Publication Date: 2010/06/01
Online ISSN: 1745-1345
DOI: 10.1587/transcom.E93.B.1431
Print ISSN: 0916-8516
Type of Manuscript: PAPER
Category: Network
vehicular ad hoc networks,  routing protocol,  Q-Learning,  AODV,  

Full Text: PDF>>
Buy this Article

In Vehicular Ad hoc Networks (VANETs), general purpose ad hoc routing protocols such as AODV cannot work efficiently due to the frequent changes in network topology caused by vehicle movement. This paper proposes a VANET routing protocol QLAODV (Q-Learning AODV) which suits unicast applications in high mobility scenarios. QLAODV is a distributed reinforcement learning routing protocol, which uses a Q-Learning algorithm to infer network state information and uses unicast control packets to check the path availability in a real time manner in order to allow Q-Learning to work efficiently in a highly dynamic network environment. QLAODV is favored by its dynamic route change mechanism, which makes it capable of reacting quickly to network topology changes. We present an analysis of the performance of QLAODV by simulation using different mobility models. The simulation results show that QLAODV can efficiently handle unicast applications in VANETs.