A Multi-Modal Fusion Method for 3-Hinge Gyrus Based on Bidirectional Cross-Attention

The 3-hinge Gyrus (3HG) is the junction of gyri from three directions during cortical folding, exhibiting unique and consistent structural and functional characteristics across individuals and species. Its number, location, and morphology can be used for quantitative analysis of cortical folding patterns. However, it is worth noting that most studies extract and quantify 3HG features on different modality respectively, which is limited in capturing the complementary information between different modalities. Thus, we propose a multimodal fusion method based on bidirectional cross-attention to integrate structural MRI and DTI data in this work. Results demonstrate that our method not only preserves the unique features of each modality, also is effective in capturing cross-modality features which are unavailable in single-modality analysis, revealing connectivity and potential functional relationships between brain regions. This multimodal fusion method may provide a further understanding on mechanisms of cortical folding patterns and brain network.