A Hierarchical Reinforcement Learning Framework for Dual-Arm Manipulation with Long-Horizon Tasks

This paper introduces a self-guided hierarchical reinforcement learning (SG-HRL) framework for complex long-horizon dual-arm manipulation tasks. Our approach addresses three fundamental challenges, including temporal credit assignment across extended horizons, state-action space explosion in dual-arm systems, and coordination between high-level and low-level networks. We decompose the long-horizon task into several sequentially executed subtasks while efficiently managing closed-chain constraints between the robotic manipulators. The proposed three-tier architecture consists of independently trained subtask policies, a self-guided neural network (SGNN) that captures successful execution patterns, and a main neural network (MNN) that enhances adaptability through reinforcement learning while preserving historical knowledge. Experimental results in the simulation environment demonstrate that our approach achieves up to 98% success rate on challenging long-horizon tasks that require precise dual-arm coordination, significantly outperforming conventional methods with 86%.