Pseudo-Text Guided Robust Learning for Noisy Correspondence in Cross-Modal Retrieval

Noisy Correspondence (NC), caused by mismatched pairs in multimedia datasets, poses a major challenge for cross-modal retrieval, especially under high noise conditions. Existing solutions often suffer substantial performance degradation as noise levels increase. To address this issue, we propose Pseudo-Text guided Robust Learning (PTRL), a novel framework designed to identifies noisy pairs and enhances model robustness. Specifically, PTRL leverages pseudo-text as explicit supervisory signals and introduces a new data division criterion to accurately distinguish between clean and noisy pairs. Rather than discarding or directly using noisy data, PTRL proposes a pseudo-text replacement strategy to maintain semantic consistency in the training set, thereby facilitating more reliable learning. In addition, pseudo-text-image pairs serve as a form of data augmentation, enriching data diversity and improving model generalization. To further stabilize training and mitigate overfitting, PTRL incorporates a robust InfoNCE loss that is particularly effective in the presence of noise. Extensive experiments demonstrate that PTRL achieves state-of-the-art performance and robustness, with an rSum improvement of +60.1% on Flickr30K and +22.6% on MS-COCO at an 80% noise level, significantly outperforming existing methods. The datasets and source codes are available at https://github.com/shidan0122/PTRL.git.