A Multi-Loss Hybrid CNN-ViT Model for Efficient Image Retrieval

Content-Based Image Retrieval (CBIR) has achieved significant advancements thanks to deep learning models, particularly Convolutional Neural Networks (CNNs). CNNs exhibit strong feature extraction capabilities, effectively capturing local image information. However, due to their limited receptive fields, CNN-based methods struggle to exploit global features efficiently, leading to incomplete image representations. In this paper, we propose an effective image retrieval approach by leveraging a hybrid feature representation combining CNN and Vision Transformer (ViT), named ML-HCVR (Multi-Loss Hybrid CNN-ViT for Retrieval). Our method fuses the local features extracted by CNN with the global features captured by ViT to generate a more robust and informative feature representation. Notably, we adopt a Multi-Loss Learning strategy that integrates Triplet Loss, Contrastive Loss, and Cross-Entropy loss to optimize feature learning, enhancing feature discriminability, and improving retrieval performance. We also present experimental results on two benchmark datasets, CIFAR-100 and CUB-200-2011, demonstrating that our proposed approach outperforms standalone models. Our findings confirm that combining CNN and ViT feature representations, along with the multi-loss learning strategy, significantly enhances feature representation capabilities, ultimately improving the effectiveness of the image retrieval system.