Bimodality Approach to Mental Disorder Diagnosis via Dual-pathway Fusion Perceptual Network

Neuroimaging technologies such as functional MRI (fMRI) are used for studying the neural mechanisms underlying mental disorders. Deep learning models offer promising solutions for analyzing neuroimaging data. However, singlemodality methods suffer from performance bottle-necks, while existing multi-modality methods fail to fully utilize the complimentary information between different modality data. We proposed a dual-pathway fusion perceptual (DP-FP) network that leverages multi-stage attentional perception, effectively distinguishing psychiatric disorder patients from healthy controls. Specifically, DP-FP consists of two pathways to receive time course and functional connection inputs respectively. Self-attentive modules were introduced into the network structure, which not only enhance the capability of fine-grained feature extraction, but also improve the interpretability of model. Furthermore, a coarse perception module and a fusion strategy were proposed, which achieved deep integration of feature representations from the two pathways and optimized the complementarity of information. We validated our proposed DP-FP network using the public schizophrenia dataset (COBRE) and the public autism dataset (ABIDE). On COBRE, the accuracy for predicting schizophrenia was found to be 87.6%. On ABIDE, the accuracy for predicting autism was found to be 71.1%. Group-discriminative brain regions were attributed and visualized through gradient-based interpretation. DP-FP focused more on the paracentral lobe and near the angular gyrus in the diagnostic task of schizophrenia, and the frontal lobe, temporal lobe and angular gyrus in the di-agnostic task of autism. DP-FP achieved state-of-the-art accuracy on benchmark datasets for both schizophrenia and autism. DP-FP also offers high interpretability, providing novel insights into exploring pathological mechanisms.