Persian Text-Image Retrieval: A Framework Based on Image Captioning and Scalable Vector Search

Text-image retrieval systems have seen significant advancements with CLIP-based models and Vision-Language models, which rely on vast datasets and powerful computational resources. However, these advancements primarily focus on English and other high-resource languages, leaving a substantial gap for languages like Persian. This paper introduces a pioneering Persian Text-Image Retrieval (PTIR), the first comprehensive framework designed to address Persian's unique challenges. We present three key innovations: (1) a diverse corpus of 1.2 million Persian image-caption pairs from multiple sources; (2) training a vision-language model combining a vision-encoder and a pre-trained Persian text-decoder to optimize cross-modal alignment and generate detailed captions; and (3) a scalable retrieval pipeline using sentence embedding models and vector databases for efficient Persian query-image matching. Evaluations on the Persian-adapted COCO benchmark dataset demonstrate PTIR achieves a 30-40% improvement over existing Persian image captioning models in standard metrics and a 9% mean improvement in Hit@K compared to the CLIP baseline. The framework's modular design enables domain-specific adaptations, offering a blueprint for low-resource language retrieval systems. A subset of our implementation, including code and datasets, is available at: https://rasoulasadiyan.github.io/PTIR.