Efficient multi-agent reinforcement learning for dexterous manipulation: a hierarchical framework with opponent modeling

With the advancement of robotic technology, the complexity and flexibility requirements for dexterous hand operations are increasing. Current research primarily focuses on utilizing single-agent reinforcement learning to achieve dexterous manipulation by robotic hands. However, challenges remain, such as the complex coordination of finger motion sequences. To address these limitations, this paper proposes a hierarchical framework with opponent modeling multi-agent reinforcement learning(HOMARL). In this approach, each finger is controlled by an independent agent, and an adversarial modeling mechanism is introduced to enhance overall collaborative efficiency. Additionally, to reduce the complexity of the action space, the motion of each finger is divided into bending and side-swaying actions. A hierarchical strategy is employed where the high-level policy selects specific actions, while the low-level policy outputs detailed parameters, such as angles. To verify the effectiveness of our method, we conducted experiments in a five-finger dexterous hand simulation environment. Extensive experiments demonstrate that our proposed method outperforms baseline algorithms in terms of performance and exhibits significant robustness.