Segment Anything Mamba: A Mamba-Based Segment Anything Model for Metallographical

Semantic segmentation has always been a fundamental issue in artificial intelligence for metallographic data analysis. However, the lack of metallographic datasets makes traditional segmentation techniques inadequate for this task. To address this problem, current research mainly adopts Convolutional Neural Networks (CNN) and uses pre-trained models, fine-tuning them based on the given data. CNNs excels in capturing local features through its convolution operations but exhibit limitations in long-range modeling capabilities. Transformers can effectively capture long-range dependencies through its self-attention mechanism whereas Transformers are hampered by their quadratic computational complexity. Recently, many models fine-tuned based on SAM (Segment Anything Model) have achieved some success in the field of medical image segmentation. However, due to the use of the ViT (Vision Transformers) network architecture in SAM's Image Encoder, there are high hardware requirements and lengthy computations during training and inference stages. At the same time, state space models (SSMs), exemplified by Mamba, have emerged as a promising approach. They not only excel in modeling long-range interactions but also maintain linear computational complexity. In this paper, we explores a new approach called Segment Anything Mamba that replaces the ViT model in SAM's Image Encoder with the Double Mamba model. Additionally, it introduces more flexible text prompts instead of the commonly used point and box prompts in the SAM model. Experimental results indicate that these changes significantly improve the model's performance and adaptability in image processing tasks.