Enriching Emotional Expressiveness in Embodied Conversational Agents Using Anime Effects

Embodied Conversational Agents (ECAs) can simulate human behaviors and express natural emotions. To further enhance these emotional expressions, we focused on applying attractive anime effects to the ECAs. Anime effects are visual techniques mainly used in anime productions, encompassing a wide range of unique types and combinations. Among these, we focused on anime effects that indirectly support and enhance the emotions expressed by anime characters. If properly applied to ECAs, it is expected to dramatically enhance their ability to express emotion. However, the absence of established methods to apply diverse anime effects to ECAs has constrained their emotional expressiveness. Therefore, we proposed a system that automatically applies these diverse anime effects to the ECAs. First, we surveyed and classified anime effects applicable to ECAs and created an anime effect dataset. Utilizing the dataset, we developed a system that automatically applies diverse anime effects to an ECA according to the ECA’s profile (gender and personality) and emotions. The proposed system was evaluated with 21 participants using two static use cases based on watching videos and with eight participants using two interactive use cases using digital signage. The results showed that the proposed system enhanced the intensity of emotional expression of the ECAs, suggesting an improvement in their interactions with users. Additionally, the applied anime effects allowed users to perceive the differences in personality between the two ECAs.