Distributed Hierarchical Reinforcement Learning for Path Planning of Long-Horizon Micro-Assembly Tasks

Assembling complex micro-devices often requires long-horizon micro-assembly tasks involving numerous precision steps. When dealing with such tasks, traditional planning methods often lack a global perspective on the entire task and find it difficult to ensure that the planned path meets complex global constraints. To address the challenges of long-horizon, multi-objective path planning in micro-assembly systems, this paper proposes Distributed Hierarchical Reinforcement Learning (DHRL) to achieve efficient and reliable path planning for precision operations. DHRL decomposes long-horizon tasks into subtasks, and combines global policy optimization to achieve multi-stage path planning that meets global constraints. In addition, the designed state-action registration mechanism ensures the accurate mapping of high-level instructions to low-level motion primitives, improving the accuracy of hierarchical decision-making. The results of the micro-assembly multi-point dispensing experiment highlight the effectiveness of DHRL: it not only successfully completes the entire long-horizon task, demonstrating its ability to cope with complex long-term challenges, but also shows better convergence characteristics and stronger exploration performance in comparison with baseline RL methods.