Multimodal demonstration knowledge guided robot skill hierarchical reinforcement learning for 3C assembly

Purpose This paper aims to address the low-efficiency issues encountered by reinforcement learning (RL) methods in standard automated assembly scenarios for 3C assembly lines, where robots struggle with vast exploration spaces to execute long-horizon sequences of actions. Design/methodology/approach This work proposes an RL framework formulated as a hierarchical reinforcement learning (HRL) approach guided by human multimodal demonstration knowledge (MDK). The framework integrates task-specific action sequences, derived from the multimodal fusion algorithm within the MDK acquisition method, into the HRL process to guide the learning. These action sequences consist of predefined action primitives, and the hierarchical policy simultaneously learns both the orchestration of these primitives and the parameters for each primitive. Findings By leveraging prior knowledge, this RL framework can effectively direct the initial learning process of the robot, significantly expediting convergence. On the other hand, the hierarchical architecture and action primitive design render its transfer to real-world robots straightforward, offering valuable references for applications in 3C assembly scenarios. Originality/value A robot learning framework that learns through human hand assembly behavior is proposed and verified, which enables faster convergence and is more convenient to be applied to actual robots.