VSS-SAM: Visual State Space-Enhanced SAM for 3D Medical Image Segmentation

The Segment Anything Model (SAM) is a large general segmentation model proposed by Meta, which has shown amazing performance in many natural image segmentation tasks. Recently, MA-SAM has been proposed to transplant SAM model to medical applications. However, such models still has shortcomings in utilizing the three-dimensional information of medical images. In this paper, we proposed a new structure to fully utilizing 3D information on medical images to enhance the performance of SAM, called VSS-SAM. Particularly, the proposed method applied two branches: 1) SAM as an encoder to learn the basic topology relations among slices, and 2) parallel Mamba as second branch to effectively capture long-range spatial dependencies. Finally, a new decoder was proposed to integrate the multi-view feature representations extracted from the two branches and output the final prediction, aiming to achieve optimal segmentation performance. We validated our method on three publicly available datasets. Experimental results show that the segmentation performance of VSS-SAM is significantly better than that of existing methods in multiple data sets.