Learning Adaptive Reserve Price in Display Advertising

Real-Time Bidding (RTB) is a trading mechanism that allocates advertising (ad) requests through online auctions. Participants in these auctions typically include an ad exchange (AdX) and several demand-side platforms (DSPs). When an RTB auction begins, the AdX first establishes the reserve price set by publishers as the starting bid, after which the DSPs bid to compete for potential ad impressions. The reserve price strategy is crucial to the ad revenue of publishers; however, due to the strategic and dynamic bidding behavior of DSPs, optimizing the reserve price presents a significant challenge. In this work, we report a novel adaptive reserve price strategy based on reinforcement learning (RL). In our scheme, value bucket identification is leveraged to estimate the intrinsic values of ad inventories. Following this estimation, specialized reward functions are utilized to generate informative reward signals for RL models. Furthermore, we study the issue of risk management on the publisher side and develop a risk-aware instantiation to model risk tendency, considering both empirical expert knowledge and real-time trading conditions. Extensive experiments using real-world datasets collected from operational environments have demonstrated the effectiveness of the proposed method.