Self-Supervised Pre-Training Based Hybrid Network for Deep Gray Matter Nuclei Segmentation

Motivation: Vision Transformers (ViTs) have the potentiality to outperform convolutional neural networks (CNNs) in the task of deep gray matter nuclei segmentation. However, Transformer-based models require large labeled datasets for training. Goal(s): Our goal is to design a Transformer-based model and alleviate the model's dependence on labeled data. Approach: We present a CNN-Transformer hybrid Network (CTNet) for deep gray matter nuclei segmentation. Moreover, a novel self-supervised pre-training approach that combines rotation prediction and masked feature reconstruction is proposed to pre-train the CTNet. Results: Our method achieves better performance than other comparison models on human brain MRI datasets. Impact: This is the first study that utilizes self-supervised learning methods for deep gray matter nuclei segmentation. Our method can achieve outstanding segmentation performance and effectively assist clinical doctors in the diagnosis and treatment of neurodegenerative diseases.