A Novel VMamba-Based Deep Learning Model for Accurate Nuclei and Cytoplasm Segmentation in Cervical Cytology Images

Cervical cancer continues to be a major global health challenge, and robust screening capabilities remain the most effective strategy for its prevention. Within this context, image segmentation plays a critical role in screening processes, as it enables a more precise analysis of cellular morphological features, thereby improving diagnostic accuracy. This paper presents a method for three-class cell segmentation into nucleus, cytoplasm and background by combining the backbone network VMamba and a transformer-based segmentation head. The model is fine-tuned on the Herlev dataset and outperforms state-of-the-art methods in cytoplasm segmentation by a moderate margin and achieves competitive above-average performance in nucleus segmentation. Across the tasks of nucleus and cytoplasm segmentation, the model yields average scores of 0.9372 for Dice, 0.8819 for IoU, 0.9217 for Precision, and 0.9551 for Recall. These results demonstrate the suitability of the proposed segmentation model to help experts effectively assess cervical cell lesions.