Investigating the Impact of Multimodal Feedback on User-Perceived Latency and Immersion with LLM-Powered Embodied Conversational Agents in Virtual Reality

Our research investigates the impact of latency on presence and immersion in virtual reality (VR) environments, focusing on interactions with LLM-powered Embodied Conversational Agents (ECAs). We explore the effectiveness of multimodal feedback strategies—including filled pauses, nonverbal turn-taking behaviours, and visual feedback—in mitigating perceived latency. Eighteen participants were subjected to both a baseline condition, without feedback interventions, and a feedback-enhanced condition. Our findings indicate that the feedback condition significantly improved the sense of presence and immersion. We also found that perceived response time and users’ impressions of the agents improved, thereby increasing willingness for future interactions. Additionally, chatbot experience positively correlated with agent likeability, whereas VR experience showed no significant correlation. These results highlight the effectiveness of feedback modalities in enhancing spatial presence and overall immersion, despite latency issues in VR interactions with LLM-powered agents.