YoloSeg: You only label once for medical image segmentation

Acquiring pixel-level annotations for medical images is an extremely time-consuming and labor-intensive task, typically occupying the majority of the development cycle for medical image segmentation models. While existing semi-supervised methods have achieved promising results, they generally still require annotations for 10%-30% of the samples to effectively guide learning from unlabeled data, which remains a substantial burden for real-world applications. In this study, we propose YoloSeg, a novel framework for medical image segmentation under extreme label scarcity, where only a single labeled image is available. YoloSeg integrates Segment Anything Model 2 to propagate labels from the labeled image to unlabeled images, thereby expanding the labeled data pool. To address the inherent noise in pseudo-labels, we employ multi-view label propagation, decomposing pseudo-labels into consensus and divergence regions. We introduce a dual-component loss to handle these regions separately, facilitating more robust pseudo-label learning for segmentation models. Additionally, we propose a cross-patch data augmentation strategy to generate new samples with stronger semantic consistency, further enhancing the stability of training and improving model generalization. We validate our method on ten diverse medical image segmentation datasets, encompassing a wide range of segmentation targets including organs, vessels, and lesions. Experimental results show that YoloSeg achieves performance comparable to fully-supervised baselines, with an average Dice score difference of only 3.08% across all tasks, and significantly outperforms other state-of-the-art semi-supervised and one-shot methods. YoloSeg significantly improves the feasibility and cost-effectiveness of deep learning in scenarios with severely limited annotation budgets. This approach holds promise for enabling the rapid development and deployment of custom segmentation models across diverse medical centers, thereby supporting the broader adoption of intelligent medical technologies. Code is available at https://github.com/iMED-Lab/YoloSeg.