Data Driven Model-Free Optimal Control For Discrete-time Nonlinear System via Q-Learning

This paper develops a data-driven model-free optimal control method by integrating value iteration Q-learning with an improved Adam algorithm, enhancing the traditional gradient descent used in policy improvement. The developed algorithm utilizes two neural networks. The improved Adam algorithm effectively adjusts the learning rate during Q-value updates, addressing the slow convergence and local optima issues associated with traditional methods. Empirical data reveal superior performance of the new method relative to conventional ones, achieving smaller Q-values and faster convergence to the global optimal solution. By combining data-driven techniques with the improved algorithm, this work shows a substantial enhancement in model-free control performance, highlighting its significant practical potential.