Emotional Alignment for Human-Robot Cooperation in Musical Tasks

Abstract Current artificial intelligence (AI) studies have witnessed the widespread use of large language models (LLMs), significantly impacting human-machine interaction (HMI) by endowing machines with emotional subtleties. Despite the efficacy of LLMs, it remains a challenge for the learning-based agents to develop reliable computational emotion models, particularly for real-world human-robot teaming, due to the lack of embodiment. This paper investigates emotional alignment within physical human-robot interaction (pHRI) scenarios. We propose a theoretical framework that enables the robot to cooperate with a human pianist in accomplishing piano-playing tasks. By considering the human users’ emotional states, the robot regulates keystroke dynamics and fingering arrangements, crucial factors affecting the resultant musical quality. Utilizing multimodal sensors and sensor fusion, the robot can capture, interpret and infer the emotional states of the human user. Consequently, the robot’s decision-making process for action selection mirrors human-like artistic patterns.