Reinforcement Learning Based Speed and Altitude Control System for Tilting Rotor UAV in Transition Flight

Tilting rotor UAVs offer great versatility for its capability in vertical take off as a fixed wing aircraft. However, it is considerably more complex in terms of control, especially regarding the changing system dynamics in transition flight. Whereas conventional control methods require accurate mathematical models, reinforcement learning succeeds by leveraging data taken directly from its environments. In this research, we present a reinforcement learning based control for a tilt rotor UAV in transition flight using the proximal policy optimization (PPO). We propose a special reward function to track altitude and speed during the transition phase to ensure minimal altitude loss and accurate speed hold control. Furthermore, we present the agent with an additional training phase under simulated wind gust and signal delays to improve its robustness. The algorithm uses the built in Reinforcement Learning Toolbox in MATLAB with the tilting rotor UAV environment modeled in Simulink. The agent is trained with randomized initial conditions and reference signals, simulated wind gusts, and signal delays. The results show that the agent is able to achieve accurate speed and altitude tracking. Furthermore, training with the simulated wind gusts does improve the controller performance with regard to the tracking accuracy. Meanwhile, simulated signal delays helps with the smoothness of the produced action. Overall, it is shown that reinforcement learning allows for great versatility with apparent robustness, which offers better performance and flexibility than traditional tilt rotor UAV control.