MIPR: A Benchmark for Reasoning-Intensive Multimodal Intent-to-Product Retrieval with Multi-Category Solutions
Real-world e-commerce queries often express user intent without naming a target product, and a single intent may be fulfilled by products from multiple categories. However, existing multimodal retrieval benchmarks typically assume a relatively well-defined target, leaving retrieval for intents with multi-category solutions less explored. To fill this gap, we introduce MIPR, a benchmark for Multimodal Intent-to-Product Retrieval comprising 3,522 multimodal queries and a shared corpus of 30,731 products. MIPR features reasoning-intensive retrieval, multi-category solutions, and fine-grained relevance labels. Experiments show that the best embedding-based retriever achieves 49.23% nDCG@10 on the full query set, and 55.86% solution-category recall@10 on queries with multiple solution categories. Reranking improves these scores to 60.11% and 67.00%, respectively, yet substantial gaps to the candidate-pool oracle remain. These results establish MIPR as a challenging and diagnostic benchmark for evaluating multimodal intent-to-product retrieval. The data and code are publicly available at https://huggingface.co/datasets/leequeue/MIPR.
Authors
- Xiangheng Kong (ORCID: https://orcid.org/0009-0006-1691-4175)
- Manxi Lin (ORCID: https://orcid.org/0000-0003-3399-8682)
- Dan Ou (ORCID: https://orcid.org/0009-0009-9838-5343)
- Wen Shen (ORCID: https://orcid.org/0000-0003-2819-2705)
- Zhihua Wei
- Jian Ruan
- Qiang Li
- Xuan Lu
- Junjun Zheng
- Xiaoyu Shen
- Jiacheng Chen
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.48550/arxiv.2611.22509
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- preprint