Reinforcement Learning Enabled UAV Path Planning to Optimize Cellular Connectivity

Drones, especially when flown Beyond Visual Line of Sight (BVLOS), can be employed to facilitate IoT in two ways: (a) As a remote airborne IoT sensor that can gather information and immediately transfer the data to ground or cloud based servers for processing, and (b) as an airborne base-station that can provide connectivity to remote areas. Both these modes are contingent upon the drones always flying in areas of good cellular (4G/5G) connectivity, thereby necessitating careful path planning. The dynamic nature of the wireless channel combined with the base station antenna beams oriented to serve only ground terminals significantly increases the probabilities of drones losing connectivity. This paper proposes a reinforcement learning (RL) model that enables a drone in a cellular (i.e., 4G or 5G) environment to learn from the RF environment it flies in, and determine the best flight path based on its characteristics. Specifically, we employ a Q-learning approach that can be trained offline to optimize link quality, handoffs and distance, and be deployed on the drone with minimal computing requirements.