Channel Self-Attention Residual Network: Learning Micro-Expression Recognition Features From Augmented Motion Flow Images

Micro-expressions (MEs) are tiny muscular movements on the face that conceal an individual's genuine emotions. However, the micro-expression recognition (MER) task faces challenges like short duration, low motion intensity, and a scarcity of training data. To solve these problems and obtain a good recognition effect, a Channel Self-Attention Residual Network (CSARNet) is proposed for extracting micro-expression discriminative information from motion stream images with augmented local features. Firstly, based on the offset frames of micro-expressions, a local feature augmentation strategy is devised to augment the local feature representations of motion flow images, thus effectively suppressing the interference of motions that are not related to micro-expressions. Second, aiming to mitigate the risk of model overfitting resulting from the dataset's limited size, CSARNet with a lightweight backbone network structure is designed to streamline the model's complexity and decrease computation time, which also accurately extracts the discriminative information of micro-expressions across channel and spatial dimensions, enabling the effective recognition of emotions. The proposed method was extensively tested on three benchmark datasets (SMIC, CASME II, SAMM) and the composite 3DB dataset, with experimental results clearly showcasing its superiority.