Dynamic facial expression recognition integrating spatiotemporal features

To address the challenges of extracting key facial features and capturing the dynamic changes of expressions in natural environments, a network model based on keyframes, named three-dimensional resnet and attention mechanism with GRU (TDRAG) was proposed. The network was capable of effectively mining the spatiotemporal information of video sequences. Firstly, it employed redundancy coefficients to select keyframes for reducing the interference of repetitive information. Secondly, three-dimensional residual attention blocks were designed to enhance the network's focus on key spatial areas of expression sequences, enabling the learning of robust facial features with occlusions and pose variations. Lastly, gate recurrent unit (GRU) unit was utilized to heighten the model's sensitivity and interpretative ability regarding temporal dimension changes, fostering a deeper understanding of the dynamic evolution of expression sequences. Experimental results demonstrate that the TDRAG model shows improvements of 4.27% in weighted average recall (WAR) and 4.16% in unweighted average recall (UAR) on the DFEW dataset, compared to the baseline model 3DResNet18, validating the effectiveness of TDRAG in extracting key facial features and enhancing the accuracy of dynamic facial expression recognition.