More Attention for Human: A Multimodal Data-Driven Human Intention Identification Method^*

"Human-Machine Symbiosis" is a defining characteristic of Industry 5.0, where human-centric manufacturing paradigms emphasize the integration of advanced production tools with intrinsically human problem-solving capabilities. While product manufacturing is inherently inseparable from product design, and conceptual design influences the cost during subsequent manufacturing stages. Current assistive manufacturing software is limited to passive command execution during interactions, lacking the ability to recognize human intent, which leads to barriers in human-machine collaboration. To address this limitation, a multimodal data-driven human intention identification method is proposed. This methodology employs 3D spatial modeling and verbal analysis to capture multimodal data generated during collaborative manufacturing processes. A novel Transformer architecture integrating T2T-ViT and Bert (TB-Multiformer) is developed to identify human intention. The multimodal features of data are extracted by inter-modal attention module and intra-modal self-attention module. A coordinated manufacture case involving two types of mechanism structures is utilized to verify the feasibility and viability of proposed method.