HAFO: A force-adaptive control framework for humanoid robots in intense interaction environments

Reinforcement learning (RL) controllers have made impressive progress in humanoid locomotion and light-weight object manipulation. However, achieving robust and precise motion control under intense force interactions remains a significant challenge. To address these limitations, this paper proposes HAFO, a dual-agent reinforcement learning framework that concurrently optimizes both a robust locomotion strategy and a precise upper-body manipulation strategy via coupled training. A constrained residual action space is employed to improve the stability of dual-agent training and enhance sample efficiency. The external tension forces are explicitly modeled using a virtual spring-damper system, allowing for fine-grained force control through manipulation of the virtual spring. In this process, the reinforcement learning policy autonomously generates disturbance-rejection responses by utilizing environmental feedback. Furthermore, HAFO employs the asymmetric actor-critic framework in which the critic network’s access to privileged external forces guides the actor network to acquire generalizable force adaptation for resisting external disturbances. The experimental results demonstrate that HAFO achieves whole-body control for humanoid robots across diverse force-interaction environments using a single dual-agent policy, delivering outstanding performance under load-bearing and thrust-disturbance conditions, while maintaining stable operation under rope-suspension conditions. Project website: https://hafo-robot.github.io .