ParaTalk: A Real-Time Paralinguistic Dialogue System for Human-Agent Interaction

Semantic-free utterances (SFUs) intuitively convey emotions and intentions, making them a promising tool for human-agent interaction. However, existing research has not addressed the challenge of generating context-aware SFU responses in real-time, limiting the adaptability and diversity of conversational systems. To address this gap, we propose ParaTalk. This real-time dialogue system dynamically generates six emotions (Happiness, Sadness, Fear, Anger, Surprise, Disgust) and five intents (Affirmation, Negation, Unsure, Question, Acknowledgement) based on user context. A preliminary user study demonstrated that ParaTalk effectively conveyed simple information through very short sounds (1-2 seconds), and maintained high user engagement. This suggests that SFU-based communication is feasible and demonstrates its potential effectiveness, particularly in scenarios where agents are required to exchange simple information.