Toward Industry 5.0: Evaluating Multimodal Virtual Human Interaction for Smart Healthcare in Simulated <scp>VR</scp> Environments

ABSTRACT To support the human‐centric goals of Industry 5.0, this paper proposes a modular framework for constructing low‐cost, high‐efficiency digital humans by combining retrieval‐augmented generation (RAG), large language models (LLMs), and AIGC (AI‐generated content). The framework enables embodied agents capable of reliable reasoning, contextual alignment, and expressive interaction across industrial environments. As a representative application in Industry 5.0 smart healthcare, we deploy three variants—scripted, LLM‐only, and LLM + RAG—in a VR‐based hospital triage simulation, integrating automatic speech recognition, semantic retrieval, neural speech synthesis, facial animation, and gesture generation. A within‐subject user study ( n = 45) evaluates task accuracy, perceived naturalness, and response latency. Results show that the LLM + RAG agent significantly outperforms others in both task success (95.1%) and naturalness rating (4.61/5), as assessed via expert consensus and standardized Likert‐based user ratings. These findings demonstrate that retrieval‐enhanced digital humans can combine factual precision, real‐time responsiveness, and multimodal expressiveness—key requirements in high‐stakes, affect‐sensitive domains. While healthcare is the tested use case, the architecture and evaluation protocol offer a reusable foundation for Industry 5.0 applications more broadly, including frontline services, education, and multilingual teleconsultation. The study contributes both a validated design pathway and a repeatable evaluation method for deploying scalable, trustworthy virtual agents in real‐world industrial systems.