Time Slot Bidding Optimization Strategy Based on TD3 in Real-Time Bidding

Real-time bidding (RTB) is a key component in digital display advertising. During the auction process, advertisers strive to maximize the total value of their winning impressions within the limited budget constraint. However, due to the complexity and volatility of the bidding environment, it is difficult to obtain an optimal bidding strategy. To solve this challenge, the advertising campaign cycle is divided into different time slots to better control and manage advertising delivery. Subsequently, the dual-delay deep deterministic policy gradient (TD3) algorithm is employed to learn the optimal bidding factor for each time slot. This approach allows for the dynamic adjustment of bidding strategies at different time slots, adapting to market fluctuations and target audience behavioral patterns. The analysis shows that direct rewards from the bidding environment were misleading. So the new reward function based on the generalized second pricing (GSP) mechanism, designed to learn optimal policy effectively. Finally, experiments on RTB dataset show that the algorithm could achieve more rewards in different bidding environments.