RESAMPL-UDA: Leveraging foundation models for unsupervised domain adaptation in biomedical images

Large annotated datasets and new models have led to significant improvements in supervised semantic segmentation. On the other side, Unsupervised Domain Adaptation (UDA) for Semantic Segmentation is still an arduous open research topic. While new ideas frequently come out based on recent findings, best methods still rely on basic techniques such as the use of pseudo-labels on target for self-training. Nonetheless, such methods fail when applied to difficult UDA cases like Biomedical Images where the domain shift is too high, leading to pseudo-labels of poor quality. In this work, we propose RESAMPL-UDA ( RE fined SAM -based P seudo L abel UDA ), an unsupervised domain adaptation method that effectively integrates zero-shot predictions from the Segment Anything (SAM) model. Given the high complexity and variability of biomedical images, SAM alone often produces detailed segmentations without necessarily capturing the intended structures. To address this, our method involves training a dedicated refinement network on source domain data to selectively enhance SAM-generated masks. These refined segmentations then serve as reliable pseudo-labels within our UDA framework, significantly facilitating the adaptation process. Experiments on 8 adaptation cases demonstrate that our method outperforms the state of the art. In addition, we extend successfully our work to Source-Free Unsupervised Domain Adaptation, demonstrating its versatility. The code will be made available. • We leverage Segment Anything as an unsupervised shape extractor for domain adaptation. • A label refinement network is trained on source data and frozen to refine on target. • This produces consistent pseudo-labels, enhancing unsupervised domain adaptation • Our method is theoretically grounded by the covariate shift assumption. • We achieve state-of-the-art results on 8 unsupervised domain adaptation cases.