From SAM 1 to SAM 3: Benchmarking Zero‐Shot Cross‐Domain Medical Image Segmentation

ABSTRACT Segmentation models have demonstrated significant potential in medical image segmentation. However, there is currently a lack of systematic, cross‐generation comparative evaluations to assess whether the iterations from SAM1 to SAM 3 can effectively enhance the clinical applicability of zero‐shot segmentation. To address this issue, this paper systematically compares the zero‐shot segmentation performance of the SAM family across four representative medical imaging datasets which cover different imaging modalities. The evaluation framework encompasses multi‐dimensional metrics and introduces logarithmic‐domain HD95 analysis to capture and quantify boundary spillover behaviour. This paper demonstrates that the evolution from SAM1 to SAM 3 represents a qualitative leap rather than a linear accumulation; SAM 3 achieves a several‐fold increase in Dice score over SAM1 on most datasets, whilst effectively suppressing boundary spillover into background; conversely, SAM 2 shows no significant advantage over SAM1 in our benchmarks. Concurrently, the zero‐shot capabilities of the models exhibit a distinct modality gradient. Only in a few modalities can SAM 3 maintain acceptable segmentation results, while in all other modalities the entire SAM family consistently fails. In summary, SAM 3 demonstrates zero‐shot clinical applicability in specific modalities. Its semantic prompts are able to segment medical images more easily and accurately than visual prompts.