Behavioral characterization of dynamic facial expression perception in rhesus monkeys (Macaca mulatta) using naturalistic and synthetic stimuli

Abstract Facial displays are an important channel of communication for primates, yet it remains unclear based on which criteria monkeys evaluate facial expressions. We trained two rhesus macaques to categorize videos of four facial expression types (neutral, lip-smacking, silent bared-teeth (SBT) and open-mouth threat displays) and then tested generalization to novel individuals and avatars while measuring pupillary responses. The animals were able to categorize facial expressions and generalized their classifications to novel, untrained videos, although performance was not perfect. Misclassifications were not driven solely by visual similarity between expression categories, as demonstrated by a novel automated approach integrating deep learning-based motion tracking with the Macaque Facial Action Coding System to enable objective quantification of facial action units. Instead, open-mouth threats were readily categorized correctly and distinguished from lip-smacking, while also eliciting the strongest arousal responses. By contrast, categorization of SBT displays varied substantially across stimulus identities, influenced by body weight, gaze direction, and coordinated movements of eyebrows and ears. Morphed avatar expressions were categorized according to expression component intensity, demonstrating graded perception. Avatar manipulations revealed that categorization was robust against lack of coherent motion and lack of facial realism beyond a basic level of texture. However, human facial expressions elicited no systematic categorizations or differential arousal. Our findings indicate that conspecific facial expressions were perceived as functionally meaningful, context-dependent social signals shaped by all components of facial behavior and signaler characteristics, rather than fixed, mouth movement-defined categories.