MedSAGa: Few-Shot Memory Efficient Medical Image Segmentation Using Gradient Low-Rank Projection in SAM

Large-scale medical image segmentation requires abundant annotated data and high computational resources which are challenging to obtain in resource-poor settings. In this study, we introduce MedSAGa: Medical Segment Anything Model with Gradient Low-rank projection (GaLore) optimization. We adopt Segment Anything Model (SAM) to achieve memory-efficient, few-shot medical image segmentation by utilizing the low-rank projection of its gradient matrices, a technique known as GaLore. We apply GaLore only on the image encoder of SAM while optimizing the mask decoder and prompt encoder using standard optimizers without GaLore. We evaluate MedSAGa's few-shot learning capabilities across multiple medical image segmentation datasets, highlighting its memory efficiency and segmentation performance. Experiments demonstrate that MedSAGa utilizes substantially low GPU memory averaging 66% less compared to baseline models. Its combination of reduced memory requirements and comparable performance positions MedSAGa as an optimal solution for resource-constrained settings.