Phased-Training Proximal Policy Optimization for UAV Hovering Stability During Suspended Payload Descent and Ascent

Designing a controller to stabilize a quadcopter with a suspended payload during hovering poses significant challenges, particularly when cable length changes during ascent and descent. These variations induce substantial changes in tension, resulting in nonlinear dynamics with abrupt changes that conventional controllers struggle to manage without detailed system modelling or specialized sensing. This paper introduces a Phased-training Proximal Policy Optimization (PT-PPO) framework that eliminates the need for explicit dynamic models to develop the controller. By gradually exposing a reinforcement learning (RL) agent to diverse initial conditions within a MuJoCo simulation, the proposed method enables the policy to develop robust control strategies across a wide range of time varying rope-length transitions. Empirical results demonstrate that the trained policy effectively stabilizes the UAV near its hover position, even under significant variations (±50%) in payload rope length, and ascent/descent rate. These findings confirm that a single, integrated RL policy can successfully manage the complex dynamics of a quadcopter-payload system without relying on multiple fine-tuned controllers or intricate mathematical formulations.