Heuristic-enhanced Proximal Policy Optimization Algorithm for Navigation

The challenge of navigating unmanned aerial vehicles (UAVs) can be effectively tackled through the application of reinforcement learning (RL) methodologies. Nonetheless, the baseline Proximal Policy Optimization (PPO) algorithm faces significant hurdles in achieving efficient convergence, primarily due to the sparse nature of rewards associated with navigation tasks. Addressing this issue, this paper presents an enhanced approach by integrating heuristic exploration strategies into the PPO framework, leading to the development of the AS-PPO (Action Switching PPO) algorithm. Furthermore, the research introduces specifically tailored reward functions designed for navigation purposes. Empirical evidence from experimental outcomes confirms the viability and efficacy of the proposed ASPPO method, highlighting its superior performance in handling continuous action spaces within navigation tasks.