Efficient learning for active wrist compensation strategy of ledge-climbing robot

Ledge-climbing robots face challenges in maintaining stability during continuous locomotion, particularly due to yaw deviation caused by gripper slipping during gripper exchange. This deviation accumulates over time and can cause the robot to fall. In this work, we propose a robust locomotion framework that integrates Central Pattern Generators (CPG) with Reinforcement Learning (RL) to enable stable multi-cycle locomotion by compensating yaw-induced lateral drift. We utilize CPG to generate a base rhythmic trajectory and train a residual RL policy to actively compensate for uncertainties. Specifically, we introduce an active wrist compensation mechanism where the RL agent learns a reference-free policy to regulate the wrist joint, correcting yaw deviation by minimizing lateral gripper displacement. To ensure efficient and smooth learning, we employ Early Stopping Policy Optimization (ESPO) combined with Generalized State-Dependent Exploration (gSDE). Training converges in 2 hours on an NVIDIA RTX 5070 Ti using 4096 parallel simulation environments. Simulation results demonstrate that our approach successfully achieves robust continuous locomotion, preventing falls where the open-loop CPG baseline fails after approximately 2000 steps. Furthermore, we found that training with wider friction randomization extends the operational friction range to lower coefficients while simultaneously improving locomotion speed by 21% at nominal friction levels. Robustness tests in simulation under sensor noise, actuator latency, mechanical backlash, and sensor drift show that the policy remains effective at moderate perturbation levels, with performance loss at higher levels. The code used in this study is available at: https://github.com/machiningman/RL-wrist-compensation .