Enhancing Trust in Intelligent Virtual Agents: The Role of Acoustic Cues, Lexical Style, and Listener Characteristics in the Perception of Synthesized Speech

Conversational agents with dialogue capabilities are becoming increasingly integrated into our daily lives.However, widespread adoption remains limited and concerns about agents' trustworthiness remain.Trust is essential for effective communication and collaboration, making it critical to understand the factors that influence trust perception in speech.This understanding can inform the design and development of conversational agents that foster trust and enhance user experiences.This paper presents a largescale, two-phase crowdsourced perception study investigating how acoustic-prosodic features (pitch, speaking rate, intensity), lexical styles (neutral, polite, humorous), and user demographics (gender, age, personality) jointly affect trust perception.Focusing on two common dialogue acts, questions and suggestions, we explore how trust is shaped by both utterance intent and delivery style, revealing that questions and suggestions impose distinct trust demands on users.Moreover, we identify context-sensitive acoustic patterns and listener-dependent responses that influence perceived trustworthiness.Our findings offer design implications for building adaptive virtual agents that modulate prosody and style to align with user expectations and conversational goals, supporting more trustworthy and engaging interactions.