Hash-Based Food Image Retrieval with Enhanced Feature Correlation Vision Transformeraper

Image retrieval is a crucial research task in the field of computer vision, and food image retrieval is an application of fine-grained image retrieval in food computation tasks. The feature extraction of food image visual spatial information is constrained by the subtle differences among various types of food. The effective combination of contextual and texture information also affects the final outcome of food image retrieval. Furthermore, differences in food processing methods introduce information errors, posing greater challenges for food image retrieval tasks. This paper is based on the pretrained Vision Transformer (ViT) model and introduces an enhanced image feature association ViT hashing network (IFCVH-Net). Targeting the associations between subcategories of food information, IFCVH-Net incorporates a weight feature integration module that uses a self-attention mechanism to assign different weights to encoded feature vectors, thereby enhancing the extraction of global features from food images. Additionally, to make full use of visual spatial information effectively, a dual-channel feature association module is designed. The IFCVH-Net model is evaluated using three popular hashing image retrieval frameworks: Greedy Hash (GreedyHash), Central Similarity Quantization (CSQ), and Deep Polarized Network (DPN). Experiments comparing the IFCVH-Net model with mainstream network models such as AlexNet, ResN<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$e$</tex>t50, ViT-B_32, ViT-B_16, EVHNet-16, and EVHNet-32 on the Food-101 and Vireo Food-172 datasets show that the IFCVH-Net model outperforms the other models in terms of retrieval accuracy.