Deep-View X-Modalities Visio-Linguistics (DV-XML) Features Engineering Image Retrieval Framework

This research proposes an advanced framework for efficient image retrieval by integrating visual and linguistic modalities into a unified system. The Deep-View X-Modalities Visio-Linguistics (DV-XML) framework is designed to handle user queries that include both text and image inputs while allowing modifications to align with user preferences. By employing a multimodal Content-Based Image Retrieval (CBIR) system, the framework combines features extracted by a ResNet-50 model for images and a Bidirectional Encoder Representations from Transformers (BERT) model for textual data. These features are harmonized using an inductive learning-based fusion methodology within Multi-Layer Perceptrons (MLPs). A novel Reverse Re-ranking (RR) algorithm enhances retrieval accuracy by optimally aligning the combined representations with the target images during inference. Extensive evaluations on the Fashion-200K and MIT-States datasets demonstrate the model's superior performance compared to baseline CBIR methods. This work advances the field by efficiently merging dual modalities and streamlining the retrieval process with innovative RR strategies, setting a benchmark for future research in multimodal image retrieval systems.