Federated Knowledge Distillation With 3D Transformer Adaptation for Weakly Labeled Multi-Organ Medical Image Segmentation

The increasing reliance on medical image segmentation for disease diagnosis, treatment planning, and therapeutic assessment has highlighted the need for robust and generalized deep learning (DL)-based segmentation frameworks. However, existing models often suffer from task-specific limitations, catastrophic forgetting, and poor scalability due to their dependency on narrowly annotated datasets. This creates a significant gap in developing unified, multi-organ segmentation systems that leverage distributed and partially labeled datasets across diverse clinical institutions. To address these challenges, we propose the Federated 3D Knowledge Distillation Network (Fed3D-KDNet), a hybrid federated learning (FL) framework that integrates both global and local knowledge distillation mechanisms. Our model adapts the Segment Anything Model (SAM) for volumetric medical imaging by introducing architectural enhancements, including 3D spatial feature adapters and an Auto Prompt Generator (APG), to optimize spatial representation and reduce reliance on manually crafted prompts. Fed3D-KDNet employs a dual knowledge distillation strategy to mitigate catastrophic forgetting and improve cross-client knowledge transfer, ensuring robust generalization across heterogeneous datasets. The proposed methodology was evaluated on multi-organ CT datasets, including the BTCV benchmark, under centralized and federated settings. Experimental results demonstrate that Fed3D-KDNet achieves state-of-the-art performance with an average Dice score of 80.53% and an average Hausdorff Distance (HD) of 11.43 voxels in federated experiments involving seven clients, showing 5.04% improvement in Dice accuracy and a 4.35 voxel reduction in HD. Moreover, our model demonstrates superior efficiency with a computational cost of 371.3 GFLOPs, 26.53 million tuned parameters, and an inference time of 0.058 seconds per iteration. These results validate the efficacy, scalability, and computational efficiency of Fed3D-KDNet, positioning it as a robust solution for multi-organ medical image segmentation in federated environments.