A pathological image-text interaction model from the perspective of state space models

With the advancement of medical sensor technology, tools for analyzing both images and text have become essential for diagnosing health conditions. Medical sensors facilitate the high-precision acquisition of pathological images, yielding rich datasets for analysis. However, existing methods face challenges in capturing fine-grained features and exploring deep semantic correlations between image and text, leading to imprecise cross-modal representations. This paper presents a novel framework based on state-space models to improve granular alignment and multimodal feature extraction. The Mamba encoder, employing a dual-path convolutional system, enhances the fidelity of sensor-derived data, overcoming the limitations of traditional VIT encoders. The Interaction Feature Enhancement Module (IFEM) reinforces inter-modal representations, while the Fine-Grained Alignment Prediction Module (FGAPM) ensures precise alignment, identifying subtle variations in pathological images. This framework substantially enhances the detection of minute variations in images, advancing cross-modal learning. The model is evaluated on four public datasets—Kather, DigestPath, PanNuke, and WSSS4LUAD—and outperforms existing models (CLIP, MuDiPath, PLIP) in Recall@50, F1-Score, and Precision.