RACE: Operator Choreography for Inference Acceleration in Personalized Recommender System

Web recommender systems play a crucial role in enhancing user experience and engagement by providing personalized content suggestions. These systems often leverage Recommendation Models (RMs) that integrate dense Deep Neural Networks (DNNs) with traditional sparse components to improve accuracy. The inference process for RMs in web recommender systems is typically executed in a CPU-GPU hybrid environment to handle the computational complexity. However, existing approaches have struggled to fully optimize RM inference and achieve real-time recommendations due to the challenges posed by the intricate RM operator graphs and heterogeneous hardware architecture. This paper presents RACE, an operator choreography framework designed to accelerate RM inference specifically for web recommender systems. RACE introduces three key components: an Operator Graph Placement (OGP) module for strategic operator placement, an Inference Embedding Table (IET) for efficient GPU lookups, and an Automatic Subgraph Fusion (ASF) module to optimize GPU utilization. Evaluated in production settings, RACE achieves a 3.2× speedup in RM inference, significantly enhancing the performance and responsiveness of web recommender systems.