Context-Aware Enhancement and Transformer Network for Image-Text Retrieval

Image-text matching is an important and challenging task in the field of multimedia analysis, aimed at bridging the semantic gap between visual content and language descriptions. Although this field has significant implications for enhancing cross-modal interaction, most previous work still faces challenges in accurately aligning images with text descriptions, especially when dealing with images containing rich semantic information. To address this issue, we propose a novel context-aware image-text matching model that extracts and summarizes visual region information aligned with multiple text descriptions from a single image. Specifically, we designed an adaptive context-aware self-attention module to extract representations of visual regions and text words. By controlling the complementary semantic relationships within each modality, our model can adaptively capture contextual information for each modality. We then introduce a Transformer-based encoder layer to extract features at multiple levels and aggregate region-level features into image-level features. Finally, fine-grained word-region alignment is conducted to match image features with corresponding text features. To evaluate the effectiveness of our method, we conducted extensive experiments on two benchmark datasets, Flickr30K and MS-COCO. The experimental results show that our model outper-forms several state-of-the-art baselines in image-text matching tasks, demonstrating the effectiveness and practicality of our model in handline cross-modal retrieval tasks.