Enhancing Semi-Supervised Instance Segmentation Through SAM-Driven Pseudo-Label Generation in Autonomous Driving Environment

In recent years, relying on large amounts of labeled data in specific scenarios, instance segmentation has made great strides in autonomous driving. However, labeling sufficient images for each target domain scene is time-consuming and impractical. Based on this, for effectively utilizing a large amount of unlabeled data in the target domain to improve the model performance, this paper proposes a Segment Anything Model (SAM)-driven Pseudo-Labels gEneration (SAMPLE) framework for enhancing semi-supervised instance segmentation (SSIS) in autonomous driving environments. This framework is the first to apply the visual foundation model SAM to an SSIS framework, using precise prompts and the segmentation capabilities of SAM to generate pseudo-labels. Additionally, to mitigate pseudo-label noise, a Multi-Point Enhancement (MPE) prompting method is proposed, which enables SAM to generate high-quality pseudo-labels and thus improve the performance of the segmentation model. Experiments were conducted on the public datasets Cityscapes and COCO. The results show that using only 5% of the labeled data from Cityscapes, the proposed method achieves a 1.8% improvement over the current state-of-the-art GD method. On COCO, SAMPLE consistently surpasses other methods across various data percentages. Furthermore, compared to the Mixture-of-Experts (MoE) method, our method boasts higher accuracy, reduces model size by 8@, decreases GPU memory usage to 1/3 of the original, and increases inference speed by 110@. These improvements lay the foundation for effectively deploying segmentation models in the autonomous driving environment.