VLGRA: Visual-Language Semantic Optimization and Graph Reasoning for cross-modal retrieval

Visual redundancy and semantic ambiguity in patch-based vision transformers significantly hinder fine-grained image-text matching. This paper proposes VLGRA, a Visual-Linguistic Semantic Optimization and Graph Reasoning Architecture, to address these challenges. VLGRA introduces two synergistic modules: a Semantic Weight Filter (SWF) that dynamically selects text-relevant patches via language-guided differentiable sampling, and a Semantic Spatial Refiner (SSR) that aggregates fragmented patches into semantically coherent visual tokens. A Relational Graph Reasoner (RGR) then performs higher-order reasoning over local–global similarity nodes. On Flickr30K, under the Faster R-CNN + Bi-GRU setting, VLGRA achieves 82.6% R@1 (image → text) and 62.5% R@1 (text → image), outperforming NAAF by 0.7 and 4.0 points, respectively. On MS-COCO 1 K, we obtain 77.5%/65.9% R@1, outperforming previous state-of-the-art methods. Extensive ablations demonstrate the effectiveness of each module, with the full framework achieving a 31.5-point improvement in rSum over the baseline.Our code is available at https://anonymous.4open.science/r/VLGRA-4921/