Micro-expression Recognition Based on Capsule Network

Since micro-expressions(ME) cannot be disguised and reflect an individual's true inner emotions, micro-expression recognitions has applications in many fields such as criminal interrogation, clinical diagnosis, and human-computer interaction. Although research on micro-expressions recognition has achieved certain results, micro-expressions have characteristics such as low movement intensity, short duration, and difficulty in capture. Additionally, existing public datasets have issues like small sample sizes and imbalanced class data, which pose significant challenges to micro-expression recognition research. In response to the characteristics of micro-expressions and the advantages of capsule networks, this paper proposes the RSR-CapsNet capsule network model. This paper proposes the RSR-CapsNet capsule network model based on the characteristics of micro-expressions and the advantages of capsule networks. The model uses SE-Res2Net as the backbone network to extract features of micro-expressions. To pay more attention to position information with significant weights, the model also incorporates a Spatial Attention Module (SAM). Due to the imbalanced sample distribution in micro-expression datasets, which is prone to overfitting, this paper proposes a residual capsule network module. Additionally, the original dynamic routing mechanism has been optimized to accelerate the computation of the routing algorithm. The proposed model first obtains vertex frames from the micro-expression sequence and then inputs the preprocessed vertex frames into RSR-CapsNet for feature extraction and clas-sification. Comparative and ablation experiments are conducted on public datasets CASME II, SMIC, SAMM, and a composite dataset. Compared to the baseline capsule network RES-CapsNet, the proposed model improves UF1 by 0.0344, 0.0261, 0.0269, and 0.0183 on CASME II, SMIC, SAMM, and the composite dataset, respectively, and improves UAR by 0.0224 and 0.0132 on CASME II and SAMM, respectively. Furthermore, the time complexity of the routing algorithm for the baseline RES-CapsNet is O(nm), while the time complexity for the RSR-CapsNet routing algorithm is O(n+m), demonstrating the effectiveness of the optimized dynamic routing algorithm.