What do the Face and Voice Reveal? Investigating Trust Dynamics During Human-Robot Interaction

Existing research has shown that vocal and non-vocal human cues correlate with human trust and distrust behaviours, suggesting their potential to measure human trust in robots in real-time. However, there is a lack of research in Human-Robot Interaction that integrates vocal and non-vocal cues into a comprehensive model to measure trust. This paper aims to estimate human trust in robots by examining vocal and non-vocal cues differences between trust and distrust states across multiple sessions of collaborative game-based HRI with 40 participants. Our analysis revealed that vocal and non-vocal human cues can indeed predict trust in HRI, with certain facial expressions, facial movements, and pitch being significant factors. Random Forest classifier achieved the highest accuracy (84 %) in classifying trust states, with key features such as facial expressions (fear, angry), facial blendshapes (cheekSquintRight, jawRight), and vocal characteristics (Duration, Harmonicity std) being the most predictive of trust. These findings demonstrate the importance of combining vocal and non-vocal cues for accurate trust measurement and highlight the potential for real-time trust assessment in robotic systems.