Reinforcement Learning-Based Control Strategies for Quadruped Robots in Challenging Terrain Environments

Quadruped robots are extensively used in tasks such as outdoor rescue and power inspection due to their flexibility and multi-legged stability. Maintaining the balance of quadruped robots while traversing complex and rugged terrains is crucial. This paper proposes a robust control method that uses the Proximal Policy Optimization (PPO) algorithm to train control strategies. Compared to the training methods in existing literature, a more innovative curriculum is proposed to train the quadruped robots. In addition to terrain updates, the challenge and impact of noise on the training performance are considered. The ratio of random noise is adaptively adjusted based on the training results at each cycle. This approach allows the quadruped robot to handle more disturbances in simpler terrains while prioritizing terrain traversal in more challenging environments before gradually increasing the noise ratio. Multiple novel randomized terrains have been added to improve the generalization ability of the quadruped robot during training. The training process employs parallel multi-robot training in Isaac Gym, significantly reducing the time required for training. The results show that under the same reward function, the proposed training curriculum yields higher reward values within the same training period. Additionally, the average terrain difficulty level the robot encounters is higher. The loss between the robot's linear and angular velocity and the target values is lower. This indicates that the proposed training curriculum enables the control strategies of trained quadruped robots to achieve better robustness and stability when navigating complex and variable rugged terrains.