Reinforcement-Learning-Based Geographic Routing Considering Future Evolution of Link States for UAV Networks

Achieving autonomous and reliable unmanned aerial vehicle (UAV) swarm applications requires a flexible and efficient communication network structure. Unfortunately, the high-speed movement of UAVs leads to drastic changes in wireless links and topology structures, posing significant challenges to reliable data transmissions. Geographic routing protocols exhibit better adaptability to highly dynamic network topologies and have garnered extensive attention in UAV networks. However, existing works did not effectively address the impact of factors such as link state fluctuations and routing holes on the performance of these protocols. To this end, by considering future evolution of link states, this paper proposes a reinforcement-learning-based geographic routing protocol (Evo-QGeo) and introduces a new routing hole bypass method. Thanks to the evaluation of future evolution of link states and the multihop optimization capability of reinforcement learning, the end-to-end packet reception rate of Evo-QGeo is improved by up to 11.81~44.61% compared to existing ones. Meanwhile, the energy consumption is reduced by up to 36.94~74.47%, the latency is reduced by up to 21.63~38.68%, and the end-to-end expected transmission count is reduced by up to19.60~26.10%. This makes Evo-QGeo more suitable for highly dynamic UAV networks.