DST-M2D: A Dualistic-Series Transformer Framework for Major Depressive Disorder Analysis Using Multimodal MRI Data

Major Depressive Disorder (MDD) is a widespread and devastating mental health disorder marked by ongoing feelings of sadness, a diminished interest in activities, and various cognitive and physical challenges. Many existing models rely exclusively on either structural MRI (sMRI) or resting-state fMRI (rs-fMRI) data, limiting their capacity to fully capture the complexity of brain activity and dynamics, which hampers our understanding of mental health conditions. Traditional methods often employ basic attention mechanisms that fail to grasp intricate relationships in high-dimensional neuroimaging data, leading to deficient feature representations. Furthermore, these approaches frequently overlook temporal variability, which is essential for understanding the fluctuations associated with mental health disorders. Conventional models also struggle with effective strategies for integrating information from different modalities, often lack interpretability, and tend to be computationally intensive, which limits their practical use. To address these challenges, this study introduces a novel framework for detecting and classifying MDD using deep learning (DL) techniques that use both sFMRI and rs-fMRI data. Our framework employs the Dualistic-series Transformer (Dual-Trans), which features an enhanced mechanism for capturing and representing features. This includes multi-Dconv-head Temporal Attention (MDCTA) for simultaneous spectral feature extraction and Multihead Spatial Self-Attention (MSPSA) for analyzing spatial relationships within the data. Additionally, the Feature Compensation Module (FCM) assesses consistency across spatial and temporal dimensions, identifying discrepancies between modalities to enhance the model's reliability. Furthermore, the Gate Module (GM) fuses these features, enabling a decision-making process that weighs their relevance in the final representation. Ultimately, detection and classification occur in a Dualistic-series Feed-Forward Network, which processes input in parallel branches tailored for each modality while benefiting from integrated features.