DED-SAM: Data-Efficient Distillation towards light-weight Medical SAM via Diverse, Balanced, Representative Core-Set Selection

Knowledge distillation (KD) offers a straightforward solution to efficiently executing the Segment Anything Model (SAM) with large computation consumption in resource-constrained clinical local deployment. However, existing vanilla KD methods for SAM require utilizing large-scale datasets, leading to inefficiency in data usage and high computational distillation costs. The key challenge for SAM-based medical distillation lies in effectively sampling from huge, long-tailed medical datasets. To address this, we propose DED-SAM, a Data-Efficient Distillation framework towards light-weight medical SAM via diverse, balanced and representative core-set selection. Specifically, DED-SAM integrates hierarchical k-means clustering and representative sampling. Hierarchical k-means clustering repeatedly partitions the data into k clusters in a bottom-up manner, aiming to preserve the diversity of data samples from the original dataset while ensuring the uniform distribution among data concepts. Representative sampling selects the highest-ranked points as representative samples via the proposed ranking metric, which augments semantic distance by incorporating the decoupled estimated IoU gain. Through representative sampling, the student model can rapidly learn from the most representative samples in each cluster, thereby improving learning and data efficiency. To validate the effectiveness of DED-SAM, we curated a diverse but imbalanced dataset Med2D-Dist-320K and provide a tail data setting Med2D-Dist-Tail. Comprehensive experiments demonstrate that DED-SAM achieves comparable performance even with only 20% training data, compared to the student model trained with 100% data. With a extreme restriction of 1% training samples, DED-SAM significantly outperforms the prior arts, achieving a 7.03% DSC and 5.96% NSD improvement over random sampling, along with a 5.14% DSC and 2.66% NSD gain in the Med2D-Dist-Tail setting.