A Two-Fold Randomization Framework for Impulse Control Problems

Abstract. We propose and analyze a randomization scheme for a general class of impulse control problems. The solution to this randomized problem is characterized as the fixed point of a compound operator which consists of a regularized nonlocal operator and a regularized stopping operator. This approach allows us to derive a semilinear Hamilton–Jacobi–Bellman equation. Through an equivalent randomization scheme with a compound Poisson measure, we establish a verification theorem that implies the uniqueness of the solution. Via an iterative approach, we prove the existence of the solution. The existence-and-uniqueness result ensures the randomized problem is well-defined. We then demonstrate that our randomized impulse control problem converges to its classical counterpart as the randomization parameter [Formula: see text] vanishes. This convergence, combined with the value function’s [Formula: see text] regularity, confirms our framework provides a robust approximation and a foundation for developing learning algorithms. Under this framework, we propose an offline reinforcement learning (RL) algorithm. Its policy improvement step is naturally derived from the iterative approach from the existence proof, which enjoys a geometric convergence rate. We numerically demonstrate the effectiveness of our algorithm using a widely studied example. The results show that our RL algorithm can learn the randomized solution, which accurately approximates its classical counterpart. A sensitivity analysis with respect to the volatility parameter [Formula: see text] in the state process effectively demonstrates the exploration–exploitation tradeoff.