Progressive Semantic Consistency Towards Unsupervised Cross-Modality Medical Image Segmentation

Although deep neural networks have made great progress in the field of multimodal medical image segmentation, they are severely limited by 1) the need to use sufficient dataset labels, and 2) ignoring the fact that in current clinical practice, medical images are often analyzed in the form of a combination of multiple modalities, and that the large differences in the images between the different modalities lead to performance of the trained models when tested in a new field degradation problem. Recently the widely studied Unsupervised Domain Adaptation (UDA) methods mitigate domain shifts based on image-level and feature-level alignment. However, the above methods have little success when the available labels are more sparse. In this study, we adopt input space semantic consistency learning and output space semantic consistency learning to realize unsupervised domain adaptation and heart structure segmentation. The framework consists of three sub-networks: a style transfer sub-network, a cross-modal segmentation sub-network, and a sub-network that maintains semantic feature consistency. We perform stage-by-stage domain alignment at the image-level, at the feature-level. Specifically, We transformed the appearance of images from different domains and enhanced the ability to learn domain-invariant features by employing self-training sub-networks with semi-supervised learning, thus addressing the more realistic problem of label sparsity. Meanwhile, we train deep neural networks to maintain semantic consistency during image style transformation and semantic consistency during segmentation sub-network output to enhance the semantic recognition ability of the model. We evaluate our approach on the MM-WHS 2017 heart dataset and demonstrate the effectiveness of our method.