Enhancing stability and explainability in reinforcement learning with machine learning

Abstract. In the field of reinforcement learning, training agents using machine learning algorithms to learn and perform tasks in complex environments has become a prevalent approach. However, reinforcement learning faces challenges such as training instability and decision opacity, which limit its feasibility in real-world applications. To solve the problems of stability and transparency in reinforcement learning, this project will use advanced algorithms like Proximal Policy Optimization (PPO), Q-DAGGER, and Gradient Boosting Decision Trees to set up reinforcement learning agents in the OpenAI Gymnasium environment. Specifically, the study selected the Atari game Breakout as the testbed, enhancing training efficiency and game performance by refining reward structures and decision-making processes, and integrating interpretable models to provide explanations for agent decisions. This study has successfully developed robust reinforcement learning agents that excel in complex environments. By employing advanced algorithms like PPO, Q-DAGGER, and Gradient Boosting Decision Trees, the study has addressed issues of training instability, and improved game performance through optimized reward structures and decision processes. Additionally, by integrating interpretable models, the study has provided insights into the learned strategies of the agents, thereby enhancing decision transparency. These findings provide crucial support for the broader application of reinforcement learning in real-world scenarios and offer valuable insights for tackling other complex tasks.