Dual-Center Loss for Multi-task Emotion Analysis

In affective computing, multi-task emotion analysis that concurrently executes discrete facial expression recognition (FER) and continuous emotion regression in the Valence-Arousal (V-A) space has attracted more attention. However, these methods still face certain key challenges: namely, FER class imbalance, V-A unsteadiness, and conflicting optimization goals for tasks. To solve these problems, we propose a novel dual-center-based multi-task framework for joint emotion classification and regression. In this paper, we specifically utilize a shared feature extractor and adopt a classification center loss to enhance intra-class compactness for the FER task, as well as a semantic-aware regression center constraint to promote intra-class structural stability and semantic consistency in V-A space. The classification center loss tries to decrease the impact of data imbalance, and the semantic-aware constraint is used to guide the regression outputs to be close to the meaningful class anchors so it can help improve the consistency in continuous space. Experiment results on the AffectNet dataset show that our method increases the classification accuracy and the consistency of the V-A prediction and reduces the task conflict in multi-task learning effectively. Compared to the baseline, our model can improve the classification performance by 4.49 percentage points and make a decrease to the RMSE value in regression. These findings demonstrate the robustness and relevance of our methodology for practical emotion analysis applications.