From bounding boxes to semantic segmentation: leveraging SAM for weak supervision in remote sensing

Semantic segmentation typically requires extensive pixel-level annotations, which are costly and time-consuming to obtain. This paper investigates the effectiveness of using the Segment Anything Model (SAM) for weakly supervised semantic segmentation of aerial and satellite imagery, utilizing only bounding box annotations. We present an approach that leverages SAM to generate pseudo ground truth annotations from bounding box prompts, which are then used to train the SegNeXT semantic segmentation model on the i-SAID dataset. Our method achieves results comparable to fully supervised training, with only a 4.2% decrease in mean Intersection over Union (mIoU). These findings demonstrate the potential of foundation models to reduce annotation costs while maintaining high performance in aerial image segmentation tasks.