Efficient Video Emotion Recognition via Multi-Scale Region-Aware Convolution and Temporal Interaction Sampling

Video emotion recognition is widely used due to its alignment with the temporal characteristics of human emotional expression, but existing models have significant shortcomings. On the one hand, Transformer multi-head self-attention modeling of global temporal dependency has problems of high computational overhead and feature similarity. On the other hand, fixed-size convolution kernels are often used, which have weak perception ability for emotional regions of different scales. Therefore, this paper proposes a video emotion recognition model that combines multi-scale region-aware convolution with temporal interactive sampling. In terms of space, multi-branch large-kernel stripe convolution is used to perceive emotional region features at different scales, and attention weights are generated for each scale feature. In terms of time, multi-layer odd-even down-sampling is performed on the time series, and odd-even sub-sequence interaction is performed to solve the problem of feature similarity, while reducing computational costs due to the linear relationship between sampling and convolution overhead. This paper was tested on CMU-MOSI, CMU-MOSEI, and Hume Reaction. The Acc-2 reached 83.4%, 85.2%, and 81.2%, respectively. The experimental results show that the model can significantly improve the accuracy of emotion recognition.