Reinforcement Learning-Based Control of a Moving Mass Coaxial Monocopter Using Soft Actor-Critic (SAC) Algorithm

This paper presents a reinforcement learning-based control framework for a moving mass coaxial monocopter, a highly nonlinear and underactuated aerial vehicle that achieves maneuverability by dynamically shifting its center of mass. To address the challenges posed by its complex dynamics and limited actuation, we employ the Soft Actor-Critic (SAC) algorithm, an off-policy, model-free reinforcement learning method recognized for its stability and sample efficiency in continuous control tasks. The SAC controller is integrated into the monocular’s system to learn optimal flight actions, including pitch, roll, and yaw, under varying flight conditions and disturbances. Training is conducted in simulation through parallelized environments, allowing the agent to refine its policy via trial-and-error interactions. The proposed approach is evaluated against both traditional PID control and alternative RL methods, with results demonstrating superior performance in terms of flight stability, adaptability, and control efficiency. This study highlights the potential of SAC-based control for moving mass aerial platforms and provides a foundation for future research in learning-based UAV control systems.