Evaluating the effect of co-speech gesture prediction on Human–Robot Interaction

Robots are starting to be used in tasks involving human–robot interactions. For them to be efficient in these tasks, they must be seen as suitable interaction partners. One method to achieve this is to enable them to use proper verbal and non-verbal communication. Selecting non-verbal behaviours that appropriately complement the robot’s verbal messages is complex and requires roboticists to understand how each dimension of communication, as well as their combination, affects how the user perceives the message. In this work, we evaluated the effect of selecting appropriate non-verbal expressions given the robot’s speech on how users perceive the robot and its expressiveness. To do this, we conducted a within-subjects experiment where participants played cards with two robots — one that used a co-speech gesture prediction module for selecting its non-verbal expressions and another that used random expressions. The results showed that using the gestures predicted by our system improves the experience of participants during interactions. Specifically, participants perceived the robot using the co-speech gesture prediction module as having a higher level of agency, and as having a more coherent expressiveness. • We conducted a within-subjects that sought to evaluate how an appropriate selection of co-speech gestures has over how users perceive a social robot and its expressiveness. • Participants observed a researcher having a staged interaction with two robots, and then played a card game with them. • The addition of the co-speech gesture prediction module led to a statistically significant improvement in how users perceived the agency of a social robot. • The use of the gesture prediction module led to participants perceiving a higher coherence between the robot’s verbal and non-verbal communication when compared with a robot that uses random, neutral non-verbal gestures. • The gestures of the robot that used the co-speech gesture prediction module were perceived as being more expressive, although the differences were only marginally significant. These differences become significant among users 40 years old or younger.