Reinforcement Learning Control with Motion Constraints for Autonomous UAV Landing on a Moving USV

This paper proposes a robust reinforcement learning (RL) control policy for autonomous unmanned aerial vehicle (UAV) landing on a moving unmanned surface vessel (USV). Addressing challenges posed by internal uncertainties, external disturbances, and the dynamic nature of USVs affected by wind and waves, the approach integrates relative position and orientation constraints to ensure safe and precise landings. The UAV dynamics are modeled using Euler-Newton equations, and a hierarchical RL framework is developed, incorporating force and torque command designs based on optimal value functions approximated via critic networks. Motion constraints include a relative position constraint surface and an orientation constraint to maintain stability within a safe landing zone. Stability analysis using a Lyapunov candidate function demonstrates that the closed-loop system is uniformly ultimately bounded, with position and attitude errors converging to a small neighborhood despite disturbances. Simulation results validate the policy, showing effective trajectory tracking, bounded signals, and robust performance under windy conditions, confirming its practical applicability for autonomous UAV-USV landing operations.