Multimodal Behavior Analysis and Impact of Culture on Affect

It has been shown that emotions are learned in a cultural way and expressions are often used to help convey these emotional states. Considering this, in this work, we investigate multimodal cultural behavior differences across 6 different cultures. More specifically, we investigate head pose, action units, and facial landmarks in British, Chinese, German, Greek, Hungarian, and Serbian cultures. Along with this, we also investigate the differences along valence and arousal dimensions for these cultures. To conduct this investigation, we evaluate the SEWA multimodal and multi-cultural dataset. We find varying differences exist that are impacted by culture, context, and modality. Based on these findings, we also perform context classification that takes into account these differences in culture. We show that incorporating culture into our pipeline improves classification performance.