Multi-head Attention Mechanism and Dynamic Semantic Graph Convolution for Facial Action Unit Intensity Estimation

Facial Action Unit (FAU) intensity estimation can simply rely on subtle responses to facial changes, and thus can be used very flexibly in many domains. Due to the co-occurrence relationship between different FAUs, earlier approaches apply the attention mechanism directly to the entire face image for information extraction, ignoring the connection between global features and local features, which leads to inaccurate FAU localization and cannot flexibly reflect the changes of FAU intensity with respect to subtle face changes. In this paper, a dynamic relationship between global and local features is established between FAUs, based on which a new learning framework is proposed. Specifically, we first capture the intensity, location, and latent features of FAUs in the form of heatmap regression, and then divide the task into two branches. The first task branch leverages an attention mechanism combined with FAU position and intensity heatmaps to extract global features. The second task branch models the activation states of FAUs within latent features, enabling dynamic exploration of correlations among AU local features. These global and local features are then interactively fused to predict AU intensity levels. The proposed method’s effectiveness is validated through experiments on the DISFA and BP4D benchmark datasets, showing superior performance compared to existing approaches.