A lightweight segmentation model based on Segment Anything Model for tongue image segmentation

Tongue image segmentation plays a crucial role in intelligent of diagnosis of Traditional Chinese Medicine. Accurate, efficient, and lightweight tongue segmentation significantly improves both the quality and practical applicability of intelligent disease diagnosis models. To address this challenge, we propose TongueSAM_Lite, a lightweight and fully automated tongue image segmentation model. Based on the Segment Anything Model. Our approach employs knowledge distillation and parameter-efficient fine-tuning to develop a novel lightweight image encoder, high-parameter modules in the Vision Transformer are partially replaced with lightweight image modules, which facilitate the transfer of its feature extraction capabilities while accelerating inference speed and reducing computational resource requirements. Additionally, to eliminate manual annotation of tongue region bounding boxes, we integrate a YOLOX-based automatic Box-prompt generator, enabling end-to-end fully automated prompting and segmentation of tongue images. To validate our approach, various experiments were conducted in three datasets. The results show that compared to the original large-scale model of the Segment Anything Model, TongueSAM_Lite reduces the size of the model by 42.7% and shortens the inference time to 45.43% while retaining the near-complete segmentation accuracy of few-shot learning. TongueSAM_Lite achieves Mean Intersection over Union scores of 96.48%, 98.36%, and 97.53% in the three datasets, respectively, outperforming state-of-the-art segmentation methods. Further validation confirms that the YOLOX-based prompt encoder yields optimal performance for the generation of tongue image bounding boxes. Our proposed approach provides new research insights to advance tongue diagnosis technology of Traditional Chinese Medicine. All codes in this article are available at https://github.com/ruanqunsheng/TongueSAM_Lite .