Enhancing Image Captioning with Retrieval-Augmented Text Features and Cross-Modal Transformer

Image captioning, situated at the intersection of computer vision and natural language processing, seeks to generate captions that are linguistically fluent, accurate, and semantically rich. The heterogeneity between visual and linguistic modalities often hinders existing methods, which struggle to learn effective cross-modal mappings, leading to limited semantic expressiveness. To address this, we propose the Retrieval-Augmented Text Features and Cross-Modal Transformer (RTCT), a frame-work designed to improve caption quality. The RTCT employs cross-modal retrieval to acquire semantically relevant texts, providing valuable semantic cues. Our Dual-Stream Feature Encoder (DSFE) enhances the interaction between image and text features, boosting their representational power. Additionally, an Adaptive Fusion Attention (AFA) mechanism, integrated during decoding, effectively fuses multimodal features to capture complex cross-modal relationships, resulting in precise and semantically coherent captions. Extensive experiments on the MSCOCO dataset demonstrate that the RTCT achieves superior performance compared to existing approaches.