A Facial Expression Recognition Network Using Rebalance-Based Regulation of Attention Consistency and Focus

Facial expression datasets commonly exhibit imbalances between various categories or between difficult and simple samples. This imbalance introduces bias into feature extraction within facial expression recognition (FER) models, which hinders the algorithm’s comprehension of emotional states and reduces the overall recognition accuracy. A novel FER model is introduced to address these issues. It integrates rebalancing mechanisms to regulate attention consistency and focus, offering enhanced efficacy. Our approach proposes the following improvements: (i) rebalancing weights are used to enhance the consistency between the heatmaps of an original face sample and its horizontally flipped counterpart; (ii) coefficient factors are incorporated into the standard cross entropy loss function, and rebalancing weights are incorporated to fine-tune the loss adjustment. Experimental results indicate that the FER model outperforms the current leading algorithm, MEK, achieving 0.69% and 2.01% increases in overall and average recognition accuracies, respectively, on the RAF-DB dataset. The model exhibits accuracy improvements of 0.49% and 1.01% in the AffectNet dataset and 0.83% and 1.23% in the FERPlus dataset, respectively. These outcomes validate the superiority and stability of the proposed FER model.