Semantic Relation-Based Cross Attention Network for Image-Text Retrieval

Image-text retrieval refers to querying the text or image of another modality given an image or text, and its key lies in the ability to accurately measure the similarity between text and image. Most of the existing retrieval methods only utilize the intramodal relations of each modality or the intermodal relations of the two modalities to perform the retrieval task. To address the above problems, we integrate the intra-modal and inter-modal relations and propose a Semantic Relation-based Cross Attention Network(SRCAN). In our proposed method, we first mine the possible intra-modal associations between regions in an image and between words in a text to obtain features with semantic relationships, and then capture the fine-grained associations between segments through the cross-attention mechanism. Finally, we balance the intra-modal and inter-modal relationships to improve the performance of the model. Our proposed method is experimentally validated on two datasets, Flickr30K and MS-COCO, and the experimental results show that our method achieves superior results.