3D Multi-Modality Cardiac Image Segmentation Using Graph Neural Network Guided Topology-Aware Learning with Boundary Enhancement

Accurate segmentation of cardiac structures from multimodal cardiac images is crucial for diagnosing and treating cardiac diseases. However, existing deep learning methods primarily rely on paired multimodal images (e.g., CT and MRI) from the same patient, which limits their applicability to unpaired clinical datasets. To overcome these issues, we present a new multimodal learning framework that utilizes graph neural network (GNN) in conjunction with a novel architecture for effective cross-modal segmentation of unpaired CT and MRI cardiac images. Our framework introduces three key innovations: (1) Pool Vision GNN module (PGM), which effectively reduces the modality difference by modeling images as graph structures and exploiting the topological features of unpaired CT/MRI data; (2) Dynamic Boundary Enhancement Module (DBE) that enhances to enhance the boundary segmentation ability of cardiac interconnected substructures and effectively solve the problem of inconsistent annotations between datasets. (3) Adaptive Sharpness Aware Minimization (ASAM) optimization strategy, which combines dynamic loss weighting and parameter space sharpness-aware update to mitigate overfitting and enhance the model's generalization performance of the model on limited datasets. We evaluate the performance of the proposed method on unpaired CT and MRI images using the MM-WHS 2017 Challenge dataset. The method achieves high segmentation accuracy with Dice coefficients of 92.9% and 88.3% on CT and MRI, respectively, surpassing competing methods.