Learning-Based Parallel Control for Unknown Nonaffine Nonzero-Sum Games

In this paper, a novel nonzero-sum game (NSG) method is developed for completely unknown nonaffine nonlinear discrete-time (DT) systems, which is referred to as model-free NSG (MNSG). First, novel dynamic control laws are developed for NSGs using parallel control, namely introducing controls into feedback. Subsequently, an augmented <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSG is formulated according to the original <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSG to derive the dynamic control laws. Furthermore, we show that the control stabilities of the original and augmented <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSGs are equivalent. In the meantime, we prove that optimal control of the augmented <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSG is equivalent to near-optimal control of the original <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSG, and the Nash equilibrium of the original <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSG can be achieved. Then, a model-free learning scheme is developed to obtain the solution of the augmented <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i>-player NSG using online policy iteration, and neither using a model network to predict unknown dynamics nor off-policy reinforcement learning (RL) is needed in the scheme. Lastly, numerical analysis, including the NSG of a DT system with unknown control-nonaffine dynamics and coupled controls, confirms the correctness of our MNSG method. The associated code is available at: https://github.com/lujingweihh/Adaptive-dynamic-programming-algorithms/tree/main/model_free_nonzero_sum_games_discrete_time.