Real-Time Social Presence Modulation of Embodied AI-based Robots: An Audio-Centric Approach

Recent advancements in large language models have enabled robotic embodiment, yielding AI-driven robots with simulated personalities and social adeptness. However, modulating embodiment and presence remains overlooked. Unlike humans and animals, who instinctively adjust presence, heightened in emergencies, subdued in focus, robots operate rigidly, lacking such adaptability, like modulating vocal tone contextually. This paper proposes a framework for real-time, context-based social presence modulation in embodied AI which comprises three components: Sensor Integration, Context Identification, and Adaptive Presence Expression. In our initial implementation, we use audio-based context detection, expandable to visual and physiological cues. The system fuses CNN-based ambient sound detection, speech-to-text keyword analysis, and sentiment evaluation via a transformer pipeline, statistically fused and classifying context through a Naive Bayes model. We define three primary states—Alarmed (emergency scenario), Social ("everyday" functioning scenario), and Disengaged (no system presence scenario) and two intermediate states to address uncertainty: Alert (between Alarmed and Social) and Passive (between Social and Disengaged). Real-world testing on a robot confirmed real-time modulation of actions and speech, validating the framework’s efficacy in adaptive social presence.