E-TD3: A Deep Reinforcement Learning-based Autonomous Flight Decision-Making Method for Unmanned Aerial Vehicles

As the application of unmanned aerial vehicles(UAVs) in low-altitude airspace continues to broaden, higher requirements have been placed on their autonomous and intelligent manoeuvring and adaptive capabilities. To overcome this challenge, this paper proposes an end-to-end UAV flight decision-making method based on deep reinforcement learning, and provides a dynamic planning scheme for the mission of safely and stably avoiding the threat of environmental obstacles and tracking the target. The method is based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) framework and introduces the Gated Recurrent Unit. To further improve the exploration capability and sample efficiency of the algorithm, we integrate expert experience into reinforcement learning and thus propose the E-TD3 algorithm. We reconstructed the experience replay buffer and designed a mixed sample collection mechanism to dynamically adjust the proportion of demonstration data. Finally, we perform experimental validation on the AirSim platform.