Is segmentation solved? An evaluation of vision foundation models for head and neck tumor segmentation

OBJECTIVE: Vision foundation models (FMs) have shown strong potential for adapting to diverse downstream tasks without specialized fine-tuning. However, their performance in head and neck cancer (HNC) tumor segmentation remains insufficiently explored. This work provides a comprehensive evaluation of zero-shot and few-shot performance of five prominent FMs for HNC tumor segmentation. 
Approach: The FMs tested include SAM, MedSAM, nnInteractive, UniverSeg, and SegGPT. The evaluation of SAM, MedSAM, and nnInteractive utilized a bounding box prompting strategy. The influence of prompt precision is assessed by systematically enlarging the prompt bounding box until it reaches twice the size of the tightest bounding box enclosing the region of interest. For UniverSeg and SegGPT, the prompt consisted of example image-label pairs, and the effect of increasing the size of this set is tested. Two-factor repeated measures ANOVAs were used for statistical analysis, confirming that both the FM architecture and the prompting strategy have a statistically significant effect on segmentation performance for both primary tumors (GTVp) and metastatic lymph nodes (GTVn).
Main Results: For SAM, MedSAM, and nnInteractive larger bounding boxes lead to notable performance degradation. Moreover, MedSAM outperformed SAM and nnInteractive for GTVp segmentation, whereas SAM provided slightly better results than the competitors for GTVn segmentation. For UniverSeg, performance improved as more image-label pairs are added to the support set, while SegGPT remained unaffected by this factor. Overall, UniverSeg and SegGPT demonstrate limited few-shot capability, with low mean dice scores across all scenarios. Comparisons against supervised CNN models trained specifically for HNC segmentation showed that conventional architectures remain more stable and clinically reliable. 
Significance: These findings highlight the ongoing challenges faced by current FMs in achieving reliable zero- and few-shot medical segmentation. The results emphasize the critical need for improved medical-domain adaptation and enhanced prompt robustness to advance the clinical utility of FMs in oncology.&#xD.