Efficiency of an RL-based Algorithm for Models with anAbsorbing Set of States

Reinforcement Learning (RL) has emerged as a fundamental approach for solving sequential decision-making problems in environments characterized by uncertainty, complexity, and dynamic evolution. Unlike classical optimization techniques, RL does not require any knowledge of the model under investigation but instead learns optimal policies through interaction and feedback. This makes RL particularly suitable for real-world applications where modeling the entire system accurately is infeasible. Over the past decades, RL has achieved significant success in diverse fields such as robotics [1], healthcare [6], cybersecurity [3], and communication networks [2]. Our work presents an RL-based algorithm that applies to a wide family of models. Specifically, our model only assumes the existence of a set of states Sa which satisfies the following property: when the agent is in a state of Sa and takes any of the possible actions, the state that the environment returns also belongs to Sa. This occurs, for instance, in any epidemic model without external infections. We present an algorithm that leverages the structure of these models. Our numerical results show that our algorithm learns the optimal policy in a more efficient manner than the Q-learning algorithm.