Reinforcement Learning enhanced Graph Learning for CTR Prediction in Online Advertising

Click-through rate (CTR) prediction serves as a key functional module in various online advertising ecosystems, such as web search engines, social media, and e-commerce platforms. CTR prediction aims to estimate the probability that a user is interested in an advertisement (ad) and will click on it. Current methods typically treat this task as a binary classification problem (i.e., clicked advertisements are labeled as positive and unclicked ones as negative), which often suffers from severe label noise issues. Furthermore, user-related features (i.e., diverse user interests and online behavior information) are heavily relied upon and may suffer from the problem of feature sparsity. To address these challenges, we propose a novel model called RL-EGL for CTR prediction in online advertising. In RL-EGL, we first introduce a heterogeneous graph to model the relationships among different types of entities in online advertising platforms. Subsequently, we build an attention-based heterogeneous graph convolution network to integrate both structural relations and semantic content information for learning ad representations. To identify effective samples from the training dataset, we further design a reinforcement learning framework to model the effective sample selection process. During the training of RL-EGL, the heterogeneous graph convolution network and prediction classifier are enhanced using the selected effective samples, while the noise-robust agent is strengthened by considering the refined node representations and the performance of the prediction classifier as feedback. Through reinforcement learning, the heterogeneous graph learning model, the agent, and the prediction classifier are trained jointly and improved together. Extensive experiments on three datasets demonstrate that RL-EGL exhibits satisfactory efficiency and outperforms state-of-the-art approaches.