A Reinforcement Learning Approach for Admission Control in Mobile Multimedia Networks with Predictive Information

Jose Manuel GIMENEZ-GUZMAN  Jorge MARTINEZ-BAUSET  Vicent PLA  

Publication
IEICE TRANSACTIONS on Communications   Vol.E90-B   No.7   pp.1663-1673
Publication Date: 2007/07/01
Online ISSN: 1745-1345
DOI: 10.1093/ietcom/e90-b.7.1663
Print ISSN: 0916-8516
Type of Manuscript: PAPER
Category: Network
Keyword: 
cellular mobile multimedia networks,  admission control,  optimization,  predictive information,  

Full Text: PDF>>
Buy this Article




Summary: 
We study the problem of optimizing admission control policies in mobile multimedia cellular networks when predictive information regarding movement is available and we evaluate the gains that can be achieved by making such predictive information available to the admission controller. We consider a general class of prediction agents which forecast the number of future handovers and we evaluate the impact on performance of aspects like: whether the prediction refers to incoming and/or outgoing handovers, inaccurate predictions, the anticipation of the prediction and the way that predictions referred to different service classes are aggregated. For the optimization process we propose a novel Reinforcement Learning approach based on the concept of afterstates. The proposed approach, when compared with conventional Reinforcement Learning, yields better solutions and with higher precision. Besides it tackles more efficiently the curse of dimensionality inherent to multimedia scenarios. Numerical results show that the performance gains measured are higher when more specific information is provided about the handover time instants, i.e. when the anticipation time is deterministic instead of stochastic. It is also shown that the utilization of the network is maintained at very high values, even when the highest improvements are observed. We also compare an optimal policy obtained deploying our approach with a previously proposed heuristic prediction scheme, showing that plenty of room for technological innovation exists.