Multi energy dynamic soaring trajectory optimization method based on reinforcement learning

In addressing the issue of dynamic soaring in unmanned aerial vehicles, a trajectory optimization approach based on deep reinforcement learning is proposed. This method synergistically utilizes gradient wind energy and solar energy and incorporates obstacle constraints to simulate complex barrier environments. It employs neural networks to approximate the Gaussian pseudospectral method for solving trajectory policies. On the foundation of the trained policies, the method utilizes the twin delayed deep deterministic policy gradient algorithm for policy enhancement. This significantly boosts the real-time inference capabilities while addressing the challenges traditional optimal control algorithms face in dynamic soaring due to varying wind fields. The experiments initially validate the approach through simulation of two classic modes of dynamic soaring, followed by Monte Carlo simulations considering multiple energy sources. The results indicate that the dynamic soaring trajectory optimization method based on deep reinforcement learning achieves energy acquisition comparable to optimal outcomes within a single soaring cycle, with a 91% reduction in real-time inference decision time. Moreover, in changing wind field environments, this method demonstrates superior adaptability compared to traditional approaches.