Reinforcement Learning in Wireless Networks:

 Learning Wireless Network Association Control With Gaussian Process Temporal Difference Methods

by Nadav Aharony and Tzachi Zehavi
Supervised by Dr. Yaakov Engel

Originated as an undergraduate research project at the Electrical Engineering Department of the Technion, Israel Institute of Technology.

Abstract

The work deals with the problem of improving the performance of wireless networks through the use of association control, which is the activity of intelligently associating users with the network's access points (APs), taking advantage of overlaps in the coverage areas of the APs. The optimal solution to this problem is classified as NP-hard. We present an innovative association control method which utilizes a novel Reinforcement Learning (RL) algorithm – Gaussian Processes Temporal Differences (GPTD). GPTD, an algorithm which addresses the value function estimation in continuous state spaces, and GPSARSA, an algorithm which uses GPTD to compute a complete RL solution, were defined and presented by Engel et al. GPTD has only been tested so far on simple and theoretical problems, and there was a desire to test its behavior under the conditions of real-life problems. In this study we attempt to accomplish the two symbiotic goals of (i) proposing a solution to the association control problem and also (ii) developing a realistic testing environment under OPNET for GPTD and for RL in general.

Acknowledgments
We would like to extend our thanks to our project supervisor, Dr. Yaakov Engel, for all the hours of guiding and support that he gave us during the project's duration, and to Prof. Ron Meir, under who the project took place. We would also like to thank the supporting staff of the computer networks laboratory and the vision and image sciences laboratory, who gave effort to make sure that our resource needs for the project were satisfied.

Related Information

• Undergraduate Research Project:
  Implementing GPTD Reinforcement Learning for Wireless Networking Association Control

  • Second Prize, Technological Innovation Competition for Technion Undergraduates, 2005.

  • Final Report (PDF).

  • Final Presentation (PPS).

   

• OPNETWORK Conference Paper:
  Nadav Aharony, Tzachi Zehavi and Yaakov Engel, Learning Wireless Network Association Control With Gaussian Process Temporal Difference Methods, OPNETWORK 2005, Washington DC, Aug. 2005

  • Best Paper Award, OPNETWORK 2005.

  • Conference Paper (PDF).

  • Conference Presentation (PPS).

Last updated: July 07, 2006