Autonomous Thermalling of Gliders Using Deep Reinforcement Learning

Gliders can gain altitude by utilizing thermal up-drafts, thereby extending the flight duration and range. Typically, the keen judgment and extensive experience from pilots are required for localizing and exploiting thermal updrafts. Within this work, a deep reinforcement learning approach is proposed for the autonomous thermalling. The agent environment is created based on a high-fidelity nonlinear flight dynamics model. With keeping the pitch angle constant, the discrete action space is defined by a set of roll angle commands, which are executed by a Proportional Integral Derivative roll controller. In particular, the control policy of agent is represented by a Long Short-Term Memory network, and the reshaped reward is maximized using Proximal Policy Optimization. The flight trajectories of the trained agent exhibit the feature of spiral ascent, successfully illustrating autonomous thermalling for sparse-reward tasks. The results demonstrate the robustness, effectiveness, and applicability of our approach. The trained agent can be further deployed on a companion computer and enhanced by interacting with real-world environments.