A Control Method for Wheeled Bipedal Robots Using Improved PPO and Multidimensional Curriculum Learning

Wheeled bipedal robots offer enhanced agility, but as an underactuated system based on a wheeled inverted pendulum model, they exhibit strong nonlinearity and coupling, making traditional control methods ineffective on complex, unstructured terrain. This work proposes Adaptive Curriculum Proximal Optimization (ACPO)-a deep reinforcement learning approach for locomotion control that combines an improved PPO algorithm with multidimensional curriculum learning (MCL). First, by incorporating an adaptive clipping coefficient based on KL divergence (Importance-Weighted Clipping, IWC) and a PI-KL self-adaptive learning rate (SALR) into PPO, ACPO achieves efficient exploration in early training and stable convergence in later stages. Second, we propose an MCL framework that dynamically increases and fine-tunes reward weights, action-command complexity, observation noise, terrain difficulty, and external disturbances. Finally, large-scale parallel training and comparative experiments are conducted in the NVIDIA Isaac Gym simulation environment. Ablation studies demonstrate that, compared to baseline PPO, ACPO signif-icantly outperforms in terms of cumulative reward, episode length, orientation stability, fall frequency, velocity-tracking error, and mean joint torque-exhibiting superior robustness, energy efficiency, and cross-terrain adaptability.