MWS-SAM: SAM-based Framework for Weakly Supervised Tumor Segmentation

Tumor segmentation plays a crucial role in clinical diagnosis and treatment. However, the presence of low contrast, blurred boundaries, and complex morphology in tumor images often leads to inaccuracies in tumor target segmentation. Recent approaches leveraging the Segment Anything Model (SAM) have made significant strides in tumor image segmentation, but they typically rely on precise pixel-level annotations, which are both costly and time-consuming, presenting a major challenge for effective tumor segmentation. To address these issues, we propose MWS-SAM, a novel SAM-based tumor segmentation framework that enables model training using only bounding box annotations, significantly reducing annotation costs while maintaining high segmentation accuracy. Specifically, the Multi-scale Feature Fusion Decoder (MFD) module significantly enhances the model’s capacity to capture tumor edge information by integrating multi-scale features from tumor images. To ease the noise introduced by bounding box annotations, we propose the Prediction-Box-Supervision (PBS) module, which transforms the predicted mask into a box mask aligned with the bounding box annotations, effectively addressing potential misrepresentations due to box bias. Additionally, we implement a consistency loss among varying predictions within the same image, which successfully reduces excessive prediction diversity and enhances overall model robustness. Results obtained from four distinct tumor segmentation datasets validate the efficacy of MWS-SAM, demonstrating comparable performance to fully supervised models without the necessity for accurate annotation.