Combining Model-Based Control and Reinforcement Learning for Autonomous Helicopter Aerial Refueling

Abstract Helicopter aerial refueling is a particularly challenging maneuver because of the complex aerodynamic interaction between the helicopter, the hose–drogue, and the tanker. To address this, a control design and analysis framework for autonomous helicopter aerial refueling is presented here. The helicopter control architecture is based on standard inner and outer-loop cascaded dynamic inversion. The outer-loop dynamic-inversion-based control is augmented by a reinforcement learning (RL) controller that corrects the outer-loop commands to account for the unpredictable drogue motion. This RL corrective input and inner-loop tracking error result in imperfect dynamic inversion in the outer-loop, leading to a nonlinear residual term in the outer-loop dynamics. Hence, we derive analytical stability and performance bounds of the proposed controller in the presence of bounded drogue uncertainty, RL control actions, and imperfect inner-loop tracking. We then use these analytical expressions to design the model-based controller. Simulations in a high-fidelity environment with full-scale helicopter and drogue models validate the proposed method. These simulation results show that the proposed control strategy reduces the mean docking error from 0.26 m with the pure model-based controller to 0.08 m, demonstrating an improvement of 69% in docking error. Furthermore, the controller is shown to have a docking success rate of 88%, while adding additional disturbances from atmospheric turbulence, wind, and state uncertainty reduces the docking success rate to 70%.