Research on Cross-modal image-text Retrieval Method based on multi-module graph structure

With the rapid development of artificial intelligence technology and the widespread application of big data, the amount of media data in the real world is showing an explosive growth trend, and a large number of heterogeneous data have emerged in various industries. Because the scale of data is continuously expanding, the semantic correspondence between images and text has become increasingly important. How to efficiently retrieve in the vast amount of image-text data has become a highly concerned issue. There are still two limitations in the existing cross-modal image-text retrieval methods: 1) Different modal data have different distributions and are uneven. 2) Semantics are complex and abstract, and there is still an insurmountable semantic gap between vision and language. In this paper, we have done the following research: A cross-modal retrieval method based on multi-module graph structure is proposed, including visual, word, and phrase graph structure modules. Due to the complexity of data and the interaction between images and text not only in semantic information but also in rich spatial location information, we propose a visual module that combines image features with location features to learn the relationship between semantics and space using GCN. The diversity of text descriptions makes it difficult to learn consistent text feature representations. For this reason, we first use GCN in the word graph structure module to capture the associations between words based on semantic dependency relationships. Then, in the phrase graph structure module, GCN is used to infer the relationships between noun phrases. Finally, a gated fusion layer is used to filter and reduce redundant information interference. This method has performed well on the Mirflickr25K and NUS-WIDE datasets.