Queue-Augmented Correlation-Biased Orthogonality Loss and Implicit Selective Transformer for Facial Expression Recognition in the Wild

Facial expression recognition (FER) in the wild suffers from ambiguous facial expressions, occlusions, and backgrounds, leading to inter-class similarities and intra-class variations. Most existing methods introduce maximization and orthogonality loss functions into convolutional neural networks (CNN) to restrict inter-class samples in a mini-batch. Such methods neglect intrinsic emotion correlations, resulting in over-separation and under-separation. Besides, CNNs have limitations in capturing global spatial information due to the inductive bias property. Thus, we propose a novel FER network to reveal intrinsic emotion correlations and learn global expression-related features for high-accurate FER in the wild. In this network, first, a correlation-biased orthogonality loss based on emotion correlations separates inter-class samples towards class distances. To decouple this loss from the mini-batch and alleviate the influence of ambiguous expressions, we use a conditional queue to maintain high-quality expressions. Second, implicit selective transformers are plugged into the intermediate stages of CNN to construct long-range relationships between local regions. Particularly, this module measures the importance of local regions implicitly to eliminate occlusions and backgrounds. Third, a cross-attention fusion module uses high-level features to guide the intermediate features to be class-specific and finally fuse them for emotion recognition. The experimental results on five in-the-wild datasets demonstrate the effectiveness and superiority of our network by showing clear performance improvements over other state-of-the-art FER methods. Codes are available at https://github.com/Gabrella/R-SFT.