EMO-Mamba: Multimodal Selective Structured State Space Model for Depression Detection

Depression is a severe mental disorder that significantly affects health and quality of life, making early detection critical for effective treatment and management. In traditional clinical diagnostics, doctors and mental health professionals typically assess emotional states by observing patients’ facial expressions and listening to their speech. Changes in facial expressions can reflect emotional responses, while vocal features such as tone and pace may indicate underlying emotional distress or psychological conditions. Artificial intelligence technologies are adept at efficiently extracting and analyzing these multimedia signals. However, existing methods often overlook critical information when processing long-time series data and struggle to handle multiple modalities simultaneously. To address these issues, we propose the EMO-Mamba, which utilizes selective mechanisms and state space models (SSM) to filter important features and effectively maintain memory over long time-series. Additionally, we introduced a multimodal data fusion framework to integrate key features from various modals, thereby enhancing model performance. Evaluations on the multimodal public datasets D-Vlog showed that our method achieved accuracy of 75.54%, demonstrating its effectiveness across diverse data environments and optimizing detection outcomes.