Large-Scale Solar-Powered UAV Attitude Control Using Deep Reinforcement Learning in Hardware-in-Loop Verification

Large-scale solar-powered unmanned aerial vehicles possess the capacity to perform long-term missions at different altitudes from near-ground to near-space, and the huge spatial span brings strict disciplines for its attitude control such as aerodynamic nonlinearity and environmental disturbances. The design efficiency and control performance are limited by the gain scheduling of linear methods in a way, which are widely used on such aircraft at present. So far, deep reinforcement learning has been demonstrated to be a promising approach for training attitude controllers for small unmanned aircraft. In this work, a low-level attitude control method based on deep reinforcement learning is proposed for solar-powered unmanned aerial vehicles, which is able to interact with high-fidelity nonlinear systems to discover optimal control laws and can receive and track the target attitude input with an arbitrary high-level control module. Considering the risks of field flight experiments, a hardware-in-loop simulation platform is established that connects the on-board avionics stack with the neural network controller trained in a digital environment. Through flight missions under different altitudes and parameter perturbation, the results show that the controller without re-training has comparable performance with the traditional PID controller, even despite physical delays and mechanical backlash.