Whittle Index-Based Q-Learning for Contextual Restless Bandits: A Case Study in Email Marketing

Email marketing has become an essential tool for businesses to reach out to potential customers [16]. However, the use of mass marketing raises ethical and environmental issues. In fact, this practice tarnishes domain reputation with potential spamming and overwhelms recipients with irrelevant content. Not only does this decrease user engagement, but it also results in a negative user experience. Moreover, it contributes to significant carbon emissions [5], especially with emails containing heavy attachments. To address these challenges, we propose a novel approach that frames the problem as a Contextual Restless Bandit Recommender System aiming to target relevant user segments while adapting to dynamic environments. The objective is to offer a more effective and sustainable solution, minimizing unnecessary email transmissions and ensuring that recipients receive content tailored to their profile.