A Multi-Stage Reward-Guided Learning Method for Complex Motions of Humanoid Robots

Humanoid robots exhibit extensive application potential in complex task scenarios such as industrial assembly, emergency rescue, and household services, where they can replace humans in accomplishing challenging tasks like high-precision operations, movement in complex terrains, and human-robot collaboration. However, the inherent multi-joint coupling characteristics and high load requirements of their humanoid structure pose severe challenges to motion control. Traditional reinforcement learning (RL) methods often suffer from low learning efficiency when dealing with complex action tasks, due to the high-dimensional complexity of the action space and the strong uncertainty of environmental dynamics. To address this, a novel multi-stage reward learning framework based on deep reinforcement learning (DRL) is proposed in this paper. Specifically, complex action tasks are adaptively decomposed into several sub-phases, with individual reward functions being customized for each. The introduction of multi-objective constraints enables the dynamic balance of stability and convergence speed during the learning process. Then, domain randomization techniques are integrated into the training process, which can significantly enhance the policy’s generalization ability across different environments, thus laying the foundation for transferring simulations to real-world applications. Based on an Isaac Gym simulation platform, systematic numerical simulation verifications are carried out for typical complex action tasks such as walking and back-flip. The simulation results clearly demonstrate that the proposed method exhibits significant effectiveness and superiority in terms of motion control accuracy and learning efficiency of humanoid robots.