Bipedal Walking Outdoors with a Point-footed Robot via Reinforcement Learning

The bipedal robot can perfectly adapt to the scenes of human society without the need for additional structural design and motion planning. Although model-based control algorithms achieve multiple types of motion, the control robustness is still insufficient in the face of complex dynamic environments. Reinforcement learning (RL) methods have recently produced good results in the field of bipedal robots. In order to explore information and exploit it better in the robot state space, we introduced a transformer structure that is more sensitive to temporal information into the RL framework. Benefiting from the self-attention mechanism in the transformer, our observation scheme extracts more state features, allowing the bipedal robot to achieve robust outdoor locomotion, including bipedal walking on asphalt, marble and grass terrains, including the test of performance on the most commonly used asphalt according to quantitative data.