MFA-CNN: An Emotion Recognition Network Integrating 1D–2D Convolutional Neural Network and Cross-Modal Causal Features

<b>Background/Objectives:</b> It has become a major direction of research in affective computing to explore the brain-information-processing mechanisms based on physiological signals such as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). However, existing research has mostly focused on feature- and decision-level fusion, with little investigation into the causal relationship between these two modalities. <b>Methods:</b> In this paper, we propose a novel emotion recognition framework for the simultaneous acquisition of EEG and fNIRS signals. This framework integrates the Granger causality (GC) method and a modality-frequency attention mechanism within a convolutional neural network backbone (MFA-CNN). First, we employed GC to quantify the causal relationships between the EEG and fNIRS signals. This revealed emotional-processing mechanisms from the perspectives of neuro-electrical activity and hemodynamic interactions. Then, we designed a 1D2D-CNN framework that fuses temporal and spatial representations and introduced the MFA module to dynamically allocate weights across modalities and frequency bands. <b>Results:</b> Experimental results demonstrated that the proposed method outperforms strong baselines under both single-modal and multi-modal conditions, showing the effectiveness of causal features in emotion recognition. <b>Conclusions:</b> These findings indicate that combining GC-based cross-modal causal features with modality-frequency attention improves EEG-fNIRS-based emotion recognition and provides a more physiologically interpretable view of emotion-related brain activity.