Video-Based Emotional Reaction Intensity Estimation Based on Multimodal Feature Extraction

This paper examines the video-based emotional reaction intensity estimation problem based on multimodal feature extraction approach. Most current state-of-the-art models are based on computationally complicated emotion recognition models, which are inappropriate for practical applications. In this paper, we propose a novel model based on multimodal embedding fusion and action-unit forecasting via OpenFace toolkit. The experimental results are provided for the data from Emotional Reaction Intensity Estimation subchallenge of the fifth Affective Behavior Analysis in-the-wild Competition (ABAW). Our approach achieves the third place on the validation set, surpassing the baseline method by a 69% increase, according to the Average Pearson's Correlation Coefficient (APCC) metric.