Prototype transferred diverse features for day-night cross-domain facial expression recognition

Facial Expression Recognition (FER) encounters significant challenges due to the limited sensitivity of visible light images in low light conditions. Most existing cross-domain emotion recognition studies have focused on domain adaptation for different visible light datasets, often neglecting the problem of emotion recognition under varying lighting conditions. To address this, we introduce the first cross-domain emotion recognition study for day and night environments, emphasizing the modal migration between normal visible light and low-light emotion maps. Considering the limited number of emotion maps and the reduced emotional detail in night time imaging, we designed an attention- switching diversified feature capture module to focus on facial expressions and extract more local emotional features that aid in migration. Given the significant domain shifts between samples in different lighting environments, we further developed a prototype feature transfer module to learn modality-independent category features and reduce domain differences between visible and low-light features. To tackle the challenge of numerous low sentiment value samples in the dataset, we introduce a high-confidence blending module to filter informative visible and infrared samples for fusion, producing features that combine the styles of both domains. Compared to state-of-the-art methods in domain adaptation and cross-domain emotion recognition, our approach demonstrates superior performance and validates its effectiveness.