Transformer-Based World Interaction Modeling for Humanoid Locomotion Control

Locomotion tasks for humanoid robots are challenging, especially in complex terrains. Understanding the physical processes of robot-environment interactions is key to achieving stable walking for humanoid robots. Since there is privileged information that the robot cannot directly access, the observation states are partially observable. Previous reinforcement learning(RL)-based methods either reconstruct environmental information from partial observations or reconstruct robotic dynamics information from partial observations, but they fail to fully model the physical processes of robot-environment interactions. In this work, we propose an end-to-end reinforcement learning control framework based on world physical interaction model for Humanoid robots. Our primary innovation is the introduction of a physical interaction world model to understand the dynamic interactions between the robot and the environment. Additionally, to address the temporal and dynamic nature of these interactions, we employ the hidden layers of Transformer-XL for implicit modeling. The proposed framework can showcase robust and flexible locomotion ability in complex environments such as slopes, stairs, and discontinuous surfaces. We validate the robustness of this method using the humanoid robot in simulations, and quantitatively compare our method against the baselines with better traversability and command-tracking.