Exploring the Impact of AI-Generated Speech on Avatar Perception and Realism in Virtual Reality Environments

The representation of avatars as virtual human characters plays a pivotal role in shaping key aspects of the user experience in virtual reality (VR), including immersion, social presence, and overall enjoyment. While prior research has predominantly focused on the impact of visual and behavioral realism on the perception of such avatars, the influence of synthetic speech on avatar realism, emotional communication, and social presence remains insufficiently explored, both in offline applications and for real-time scenarios. This study systematically compares various machine learning-based speech synthesis and voice conversion algorithms with human speech across neutral and emotional stimuli to evaluate how different AI-based synthetic speech methods affect user perceptions of avatars in VR. Building upon existing literature, the investigation seeks to examine the impact of nuanced variations in synthetic auditory stimuli, with consistent timbre and speaker identity, while exploring the use of a potentially real-time voice conversion algorithm in an ecologically valid VR setting. Our findings provide valuable insights into the interplay between AI, auditory realism and avatar presence, with broader implications for live social VR platforms, therapeutic interventions, and immersive environments.