Reinforcement learning approximation for optimal active lifetime investment

This paper investigates optimal investment strategies for individuals under lifetime constraints and parameter uncertainty, with applications to insurance product design for individual capital allocation. Addressing the issue of model parameter misidentification in risk management, we propose a stochastic model that integrates entropy regularization and reinforcement learning (ERRL). The key advantage of this model is its ability to execute distributional policies while optimizing decision performance, transforming the control problem into a distributional and exploratory framework. We derive the exploratory Hamilton-Jacobi-Bellman (HJB) equation associated with the value function. When lifetime follows an exponential distribution, the HJB equation reduces to an ordinary differential equation (ODE). We develop a numerical method to solve two critical risk management problems, illustrating the effects of mortality intensity and learning rate on the value function and optimal policies. For more general mortality functions, we introduce a policy improvement algorithm (PIA) to approximate the value function and track optimal policies. A numerical example demonstrates the PIA, showcasing the impact of mortality risk, the loss function, and current age on the performance of the value function.