Graph-guided Source Selection with Sequential Transfer for Medical Image Segmentation

Transfer learning is a critical technique in training deep neural networks to tackle downstream tasks with better or faster solutions, by leveraging previously acquired knowledge. This is especially useful in the low-resource medical image analysis field. However, transferring knowledge from a less related source task can negatively impact the target task’s performance, making the selection of appropriate source tasks vital. To make up for its deficiency when applying transfer learning to medical image segmentation, we propose a novel source selection framework to identify the landmark source with an effective sequential transfer path for the given target task. Specifically, we first construct a comprehensive graph that reflects the relatedness among source tasks, using a medical-image-tailored task affinity metric. Guided by the graph, we assess the informativeness and representativeness of each node and identify the landmark source for the target medical segmentation task. To ensure a positive transfer, we pinpoint a sequential transfer path to the target by minimizing both transfer and search costs. By optimizing the use of source tasks, we consequently improve the transfer performance on the target task. Extensive experiments on three brain MRI medical datasets demonstrate the efficacy of the proposed framework in finding the best source sequence. The results show that our method outperforms other transfer learning approaches by a considerable margin, improving state-of-the-art performance by 6.61% for FeTS 2022, 0.66% for iSeg-2019, and 1.70% for WMH in terms of segmentation Dice score.