MSDA-ACLDR: Multi-source Domain Adaptation Augmented Contrastive Learning with Dual-stream Representations for Emotion Recognition

Electroencephalogram (EEG) shows significant potential for emotion recognition. However, a single EEG representation fails to comprehensively capture emotional information due to the complexity of EEG signals and the diversity of emotions. Additionally, the non-stationarity of EEG leads to significant inter-subject variability. To this end, we propose a Multi-source Domain Adaptation Augmented Contrastive Learning with Dual-stream Representations (MSDA-ACLDR) for EEGbased emotion recognition. Specifically, we first construct two different augmentation methods using EEG data to simulate two different views of EEG, enhancing data diversity. Subsequently, we develop a dual-stream representation module based on intra-view and inter-view blocks to fuse view features, which capture the correlation information between different views. Moreover, we introduce contrastive loss before feature fusion to ensure information consistency between views. Furthermore, we incorporate multi-source domain adaptation to align subjects and categories separately, which improves inter-subject similarity and reduces category differences, minimizing domain discrepancies. Moreover, we introduce conditional entropy loss to reduce model uncertainty in the target domain, forming a unified domain adaptation framework that enhances model performance through joint optimization. Extensive experiments on the SEED and SEED-IV datasets validate the effectiveness and superiority of our proposed method.