MICE: Mixture of Image Captioning Experts Augmented e-Commerce Product Attribute Value Extraction

Attribute value extraction plays a crucial role in enhancing e-commerce search, filtering, and recommendation systems.However, prior visual attribute value extraction methods typically rely on both product images and textual information such as product descriptions and titles.In practice, text can be ambiguous, inaccurate, or unavailable, which can degrade model performance.We propose Mixture of Image Captioning Experts (MICE), a novel augmentation framework for product attribute value extraction.MICE leverages a curated pool of image captioning models to generate accurate captions from product images, resulting in robust attribute extraction solely from an image.Extensive experiments on the public ImplicitAVE dataset and a proprietary women's tops dataset demonstrate that MICE significantly improves the performance of state-of-the-art large multimodal models (LMMs) in both zero-shot and fine-tuning settings.An ablation study validates the contribution of each component in the framework.MICE's modular design offers scalability and adaptability, making it well-suited for diverse industrial applications with varying computational and latency requirements.