Optimize Visual Shopping Journey with Embedding-based Retrieval in Pinterest Closeup

Pinterest is the visual discovery platform where people find inspiration, curate ideas, and shop products for all life's moments. An intentful journey can start when Pinners (users) click on Pins and arrive at the Closeup surface, where they can continue to explore or refine their intent by browsing related content powered by our visual search and related recommendation engines. Product Pins, or contents that are linked to merchants and are buyable in general, are critical to realizable fulfillment in Pinners' exploratory journey. This paper focuses on optimizing embedding-based retrieval (EBR) to retrieve relevant and personalized product Pins to drive actionable engagement. In contrast to conventional EBR systems, we introduce complementary Shopping Priority Corpora that are prepared by probability models from a dynamic inventory with billions of candidates and highly skewed distributions. This novel design enabled us to significantly improve the retrieval efficiency, scale up the systems, and meet different business requirements. On top of that, we build retrieval models that infuse multimodal information with multi-task contrastive learning to balance relevance and engagement. We evaluate the EBR system on random off-policy traffic with thorough baseline comparisons and rigorous online A/B experiments. This work leads to significant metric gains in our production systems and provides practical lessons on improving early retrieval for multiple business objectives at large scales.