Leveraging distillation in transductive learning for improved medical image segmentation

Medical image segmentation with deep learning has demonstrated considerable potential, but remains challenged by the limited availability of annotated data. Traditional supervised approaches are constrained by the need for extensive expert-labeled datasets, particularly for 3D images where annotation is time-consuming and resource-intensive. We propose TRIM (TRansductive-Inference-based Model), a novel transductive learning framework that leverages both labeled and unlabeled data to enhance segmentation performance. TRIM first generates pseudo-labels from test data using a pre-trained model, then iteratively refines itself through self-training. To address computational constraints, we introduce complementary data distillation (DD) and weight distillation (WD) mechanisms, which significantly reduce training and inference overhead while maintaining accuracy. Our approach secured 1 st place in the competitive CHAOS multi-modal segmentation challenge, achieving a mean Dice score of 94.95% across five tasks and outperforming established methods. A comprehensive evaluation on diverse 3D medical imaging modalities (MR, CT, and US) demonstrated the robustness and performance of our method, while requiring significantly fewer parameters and computing resources.TRIM enables accurate medical image segmentation with minimal expert input by combining transductive inference and efficient distillation, providing a practical solution for diverse medical imaging applications.