MRF-CMFT: Mamba-Based Quality-Gated Multi-View Fusion Framework for EEG–ECG Affect Recognition on Low-Quality Data

Abstract Physiological affect recognition is inherently a multimodal fusion problem, yet its reliability is limited by noisy measurements, heterogeneous representations , and temporally varying signal quality. We propose Multi-view Representation Fusion with Quality-aware Cross-Modal Feature Transformation (MRF-CMFT), a quality-aware framework for EEG–ECG affect recognition. Each modality is represented using three complementary views: short-time Fourier transform power maps, Morlet wavelet scalograms, and compact handcrafted feature representations. These views are encoded by convo-lutional sequence encoders with a Mamba-based backbone and integrated through adaptive quality-aware view fusion. Cross-modal attention captures interactions between EEG and ECG, while a quality-aware modality gate dynamically regulates their contributions according to estimated signal reliability. A temporal Mamba aggregator then integrates consecutive windows to produce trial-level predictions. The MRF-CMFT was evaluated on the DREAMER and AMIGOS datasets using trial-grouped subject-dependent, subject-independent, and leave-one-subject-out protocols to prevent information leakage. On DREAMER, the framework achieves subject-independent AUC of 0.640, 0.534, and 0.581 for valence, arousal, and dominance, respectively ; the corresponding AMIGOS AUC are 0.496, 0.561, and 0.623. The mean subject-dependent AUC is 0.614 on DREAMER and 0.616 on AMI-GOS. Under the evaluated corruption conditions, the quality-aware fusion mechanism exhibits graceful degradation, with dominance AUC on AMIGOS remaining above 0.83 across all tested scenarios. These findings indicate that explicitly modelling representation reliability and cross-modal complementar-ity supports robust physiological affect recognition under subject variability and degraded sensing conditions.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-11411925/v1
Primary Topic
Emotion and Mood Recognition
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

MRF-CMFT: Mamba-Based Quality-Gated Multi-View Fusion Framework for EEG–ECG Affect Recognition on Low-Quality Data

Ahmed M. Anter, Mervat Samy, Habiba Mostafa, Nedaa M. Elsherbini
Research Square
Emotion and Mood Recognition
preprint

MRF-CMFT: Mamba-Based Quality-Gated Multi-View Fusion Framework for EEG–ECG Affect Recognition on Low-Quality Data

Ahmed M. Anter, Mervat Samy, Habiba Mostafa, Nedaa M. Elsherbini
preprint en

Abstract

Abstract Physiological affect recognition is inherently a multimodal fusion problem, yet its reliability is limited by noisy measurements, heterogeneous representations , and temporally varying signal quality. We propose Multi-view Representation Fusion with Quality-aware Cross-Modal Feature Transformation (MRF-CMFT), a quality-aware framework for EEG–ECG affect recognition. Each modality is represented using three complementary views: short-time Fourier transform power maps, Morlet wavelet scalograms, and compact handcrafted feature representations. These views are encoded by convo-lutional sequence encoders with a Mamba-based backbone and integrated through adaptive quality-aware view fusion. Cross-modal attention captures interactions between EEG and ECG, while a quality-aware modality gate dynamically regulates their contributions according to estimated signal reliability. A temporal Mamba aggregator then integrates consecutive windows to produce trial-level predictions. The MRF-CMFT was evaluated on the DREAMER and AMIGOS datasets using trial-grouped subject-dependent, subject-independent, and leave-one-subject-out protocols to prevent information leakage. On DREAMER, the framework achieves subject-independent AUC of 0.640, 0.534, and 0.581 for valence, arousal, and dominance, respectively ; the corresponding AMIGOS AUC are 0.496, 0.561, and 0.623. The mean subject-dependent AUC is 0.614 on DREAMER and 0.616 on AMI-GOS. Under the evaluated corruption conditions, the quality-aware fusion mechanism exhibits graceful degradation, with dominance AUC on AMIGOS remaining above 0.83 across all tested scenarios. These findings indicate that explicitly modelling representation reliability and cross-modal complementar-ity supports robust physiological affect recognition under subject variability and degraded sensing conditions.

Research Square
Mansoura University (EG), Egypt-Japan University of Science and Technology (EG)
Good health and well-being
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon