Universal Visual Commerce Intelligence: Ingesting Short-Form Video via Adaptive Marketplace Evidence Fusion
The proliferation of user-generated visual content on short-form video platforms has created new opportunities for product discovery, yet retrieving commercial listings directly from unconstrained video streams remains an open challenge. This paper introduces Universal Visual Commerce Intelligence (UVCI), a framework for Open-World Visual Commerce Retrieval (OWVCR)—the task of identifying and retrieving marketplace products from user-generated videos without pre-aligned catalogs. At the core of UVCI is Adaptive Marketplace Evidence Fusion (AMEF), an entropy-based probabilistic evidence fusion strategy derived using maximum entropy principles. AMEF employs a probabilistic routing policy that selectively invokes cloud-hosted vision-language models only when local prediction uncertainty exceeds a learned threshold, enabling principled cost-accuracy trade-offs. We establish a theoretical framework including convergence guarantees, accuracy bounds, and complexity analysis. To support evaluation, we introduce Cartly-350, a benchmark comprising 350 manually annotated creator reels with 5,600 labeled product instances across four product categories. Within our evaluation, AMEF achieves 0.915 Precision@1, outperforming CLIP-based global search by 30.3 percentage points and reducing cloud API costs by 99.6% compared to cloud-only baselines. The code and benchmark will be released upon publication to facilitate reproducible research.
Authors
- Omkar Ahirrao (ORCID: https://orcid.org/0009-0001-5172-5437)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.5281/zenodo.24280856
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- preprint