HYBRIDHEARTGAN: A GAN-BASED FRAMEWORK FOR CROSS-DOMAIN WHOLE HEART SEGMENTATION FROM MRI TO CT

Whole heart segmentation is critical to cardiology because it allows accurate diagnosis and treatment planning. Nevertheless, due to the differences in contrast, resolution, and noise between imaging modalities, existing methods can often not generalize to modalities such as MRI and CT. Although traditional architectures like U-Net perform well in a single domain, they fail to generalize due to domain shifts. Although domain adaptation models like CycleGAN improve some properties, they trade boundary accuracy and fine anatomical details. This backdrop of cross-domain segmentation presents these challenges and motivates the need for a more comprehensive framework. We present HybridHeartGAN, a proposed GAN approach composed of a Hybrid 3D U-Net generator and a CNN-based discriminator that helps to reconcile domain discrepancies and produces segmentation masks that are inferably anatomically plausible. It combines Dice loss and adversarial loss to adversarially train a segmentation network that makes both an accurate segmentation and a realistic mask. A sequential training process periodically updates the generator and discriminator, ensuring convergence and preventing overfitting for DS conditions. Validation of the framework on MRI and CT datasets revealed DSC 91.2% for MRI and 89.7% for CT, excelling in state-of-the-art methods. Its property of handling domain shifts enables multi-modal medical image segmentation via HybridHeartGAN.