Improving Quadrotor Tracking Performance in Wind Disturbances Using Hybrid Reinforcement Learning Control

The control performance of model-based control approaches depends on the accuracy of the system dynamics. However, due to unknown environmental disturbances, quadrotor dynamics are time-varying during real-world flight, and real-time updating of the dynamics is required to avoid degradation of control performance due to model errors. To address this challenging issue, this paper proposes a hybrid controller that combines model predictive control (MPC) with model-based reinforcement learning (MBRL) to improve quadrotor trajectory tracking accuracy in unknown wind conditions. The MPC controller uses simple dynamics identified from offline flight data. The MBRL controller learns MPC-augmented dynamics represented by the Gaussian process (GP) and the residual policy from online flight data. The RL controller outputs additional actions to compensate for MPC control errors, while MPC can ensure the safety of RL exploration. The computational cost of GP when dealing with a large dataset affects the real-time performance. We design a priority data selection criterion to maintain a small dataset, balancing model accuracy and computational time. The accurate tracking results for different reference trajectories under various wind conditions demonstrate the robustness of the proposed approach. Comparative results with nominal MPC and related baselines show the advantages of the proposed approach in terms of control and real-time performance.