Bridging Exploration and Safety: A Distilled NP3O Framework for Constrained Reinforcement Learning in Legged Robots

Reinforcement learning is increasingly replacing classic model-based control methods for legged robots, becoming a mainstream approach for achieving high flexibility, adaptive decision-making, and complex behavior generation. However, practical deployment still faces critical challenges, such as low training efficiency, limited policy generalization, and lack of safety constraints. To address these issues, this paper proposes a safety-oriented reinforcement learning method based on Normalized penalized proximal policy optimization (NP3O), enhanced with a teacher-student architecture. The method not only enhances training stability but also effectively prevents unsafe behaviors during policy learning. By introducing a knowledge transfer mechanism, it mitigates policy oscillations and overfitting during training. In addition, the integration of the constraint optimization and penalization terms effectively enforces safety during policy learning by avoiding unsafe behaviors. The proposed approach is evaluated through comparative and ablation experiments and deployed on our self-developed quadruped robot SDUQuad144. Real-world experiments across various complex environments demonstrate the method’s strong generalization ability and robustness.