Interactive Content-based Image Retrieval (Interactive CBIR): Hybrid Transformer- Compressed Convolutional Model for Deep Feature Extraction

Content-Based Image Retrieval (CBIR) systems tend to have difficulties maintaining local detail extraction versus global context comprehension. CNN-based solutions are great at encoding local features but have no global perception, whereas Transformer-based ones provide global attention but can miss out on minute details. In an effort to bridge this void, this paper presents ViT-ResNet-CCN, a hybrid model that combines ResNet for local feature extraction, Vision Transformer (ViT) for global context modeling, and a Custom Combination Network (CCN) to combine these features in an optimal manner. Our approach is to concurrently extract ResNet and ViT features, align, and combine them using the CCN module for better feature representation to facilitate improved retrieval performance. We benchmark our model on benchmark CBIR data sets with appreciable improvement in precision and recall compared to sole CNN or Transformer models. This research identifies the potential of the hybrid architecture paradigm for CBIR and opens avenues to more sophisticated and semantically richer retrieval machines for real-life applications like visual search engines and digital asset management.