Enhancing Coarse-Grained and Fine-Grained Image-Text Retrieval Models via Multi-Scale Modal Distance Learning

The current image-text cross-modal retrieval faces challenges due to the heterogeneous nature of different modalities. This paper proposes an improved model based on multi-scale modal distance learning to enhance the performance of coarse-grained and fine-grained image-text matching. In the image encoding part, an improved ResNet architecture is adopted, utilizing upsampling techniques to better preserve key semantic information. For text encoding, BERT and Bi-LSTM models are combined to extract shallow and deep textual features, respectively. We design a coarse-fine granularity interaction mechanism, leveraging a bidirectional stacked Transformer to achieve local similarity interaction between image regions and text words, while also using Bi-LSTM-extracted contextual features to interact with global image features for global similarity matching. Additionally, this paper introduces a novel multi-scale modal contrastive loss, which dynamically adjusts the distance between samples of different modalities, thereby improving the model's sensitivity to semantic similarity. Experimental results on the COCO 5K and Flickr30K datasets demonstrate that the proposed method outperforms existing mainstream methods in both retrieval accuracy and efficiency, achieving complementary advantages between coarse- and fine-grained features. This provides an effective solution and new research direction for cross-modal retrieval.