A Facial Micro-Expression Recognition Method Based on Box-Attention Mechanism and MultiScale Feature Fusion

As a special form of emotional expression, microexpressions are usually inadvertently revealed in facial expressions and body movements when human beings try to hide their true emotions. However, due to the fact that characteristics of micro-expressions such as short duration and low expression amplitude, it is complicated to capture the features of micro-expressions' subtle changes, which leads to a low recognition accuracy. To address this problem, a face micro-expression recognition network model based on dual-branch multi-scale feature fusion network (DmffNet) is proposed. To begin with, the first network branch of this model is based on the constructed YOLOv5-Box network to extract the three key regions of the face, namely the left eyebrow, the right eyebrow, and the mouth, and the local features of these three key regions are extracted by Vision Transformer. The second network branch firstly obtains fourlayer multi-scale facial features by a deep neural network which combining residual network and progressive pyramid, and then obtains the global features of the face by multi-scale feature fusion based on progressive pyramid network. And then, the local and global features extracted from the two network branches are fused by the attention gate module. Finally, the fused micro-expression features are input into a classifier to implemen micro-expression recognition. The DmffNet model achieves an average accuracy of 98.71 % on the Micro-Macro-Expression Warehouse (MMEW) dataset, and the experimental results demonstrate that the model possesses a superior micro-expression recognition performance.