Enhancing Industry 5.0 Assistance in Assembling Tasks with Large Language Models and Retrieval Augmented Generation: the DIA Framework

Industrial organizations are increasingly adopting Artificial Intelligence (AI) as a cornerstone of their transition to Industry 4.0 and Industry 5.0, seeking to automate complex workflows and enable data-driven decision-making. Despite significant advancements, many critical tasks in industrial quality control still require substantial human intervention, particularly when handling dynamic and unpredictable scenarios. This paper introduces DIA (Digital Interactive Assistant), a novel Industry 5.0 multimodal digital assistant designed to support operators during assembly processes through intuitive voice-based interaction and visual analysis capabilities. The DIA system leverages Large Language Models (LLMs) to provide seamless multimodal support without requiring additional training or setup, while maintaining high-quality guidance throughout assembly tasks. The system architecture comprises four main layers: Voice Interface, Query Processing, Knowledge Retrieval, and Response Generation, enabling both document-based information retrieval and component verification through image analysis. A controlled experiment with two groups of four participants each compared assembly task performance with and without DIA assistance. Results demonstrate that while DIA significantly increases task completion time (interactions accounting for approximately 60% of total duration), it substantially reduces human error rates. The system achieved 78.91% fully satisfactory responses, with visual interactions showing superior reliability compared to voice-based interactions. Speech recognition demonstrated robust performance with an average Word Error Rate of 0.13. The findings reveal important trade-offs between operational efficiency and reliability in human-AI collaboration, highlighting the potential of multimodal AI assistants in industrial settings.