A Hybrid ResNet-ViT Architecture for Multi-Attribute Prediction in Product Image Classification

Accurately identifying the characteristics of products from supplier uploaded photos during cataloguing is essential for any e-commerce platform. Most merchants and suppliers in these platforms, particularly those with limited resources, have difficulty manually entering all the information. This procedure is sluggish, laborious, and susceptible to errors, as evident from a quick review of any e-commerce platform catalog where for instance, an image may show a short-sleeved garment while the description incorrectly states it is long-sleeved. Such inconsistencies can result in client frustration and ambiguity over the purchases. This paper proposes a solution to the issue with a hybrid model that integrates the ResNet101 and ViT model, which is then trained using the stacked ensemble learning approach to predict product features directly from images. The model’s average accuracy is 0.8548, precision is 0.7946, recall is 0.7137, and F1 score is 0.7247. The findings indicate that this attribute prediction model can enhance catalogue accuracy, save human labour, and elevate consumer experience, in real-world environments.