Dual Task Learning: A Semi-Supervised Approach to Medical Image Joint Segmentation and Registration

This work proposes a novel multi-scale attention-enhanced dual-task network, MSA-DTNet, to simultaneously address two critical tasks, segmentation and registration. MSA-DTNet is designed for high-resolution 3D MRI data to incorporate multi-scale convolutions to capture both local and global features and attention mechanisms to enhance the model’s focus on key anatomical structures. By jointly optimizing segmentation and registration tasks, our network improves anatomical consistency and overall performance in medical image processing. The segmentation decoder produces high-quality segmentation maps, while the registration decoder outputs a displacement field for aligning images with a reference. A novel hybrid loss function is also proposed to optimize the model during training. The experiments on the brain MRI dataset demonstrate that MSA-DTNet outperforms existing state-of-the-art networks in terms of dice score (DSC), intersection over union (IoU), precision and recall in segmentation, and DSC and mean squared error (MSE) in registration tasks. Our model also achieves significant performance improvements, even with limited labeled data, by leveraging semi-supervised learning.