Enhancing the Adaptability of Hexapod Robots via Multi-Agent Reinforcement Learning and Value Function Decomposition

This paper applies the value function decomposition algorithm, QPLEX, to hexapod robot motion control. The QPLEX architecture is designed for multi-agent reinforcement learning, and in the case of the hexapod robot, each leg is treated as a separate agent. The QPLEX algorithm is trained using reinforcement learning in a simulated environment to achieve stable and efficient locomotion. The architecture outperforms a fully-decentralized baseline architecture in achieving walking behavior with rapid learning convergence and results in well-coordinated, rhythmic gaits. The study examines the generalization of the trained controllers to uneven terrain conditions and shows that the QPLEX architecture is more robust and performs significantly better than the fully-decentralized approach on uneven terrain. The importance of terrain curriculum learning is also evaluated.