Data-Driven H∞ Output Consensus for Heterogeneous Multiagent Systems Under Switching Topology via Reinforcement Learning

In this article, a novel model-free policy gradient reinforcement learning algorithm is proposed to solve the tracking problem for discrete-time heterogeneous multiagent systems with external disturbances over switching topology. The dynamics of the followers and the leader are unknown, and the leader's information is missing for each agent due to the switching topology. Therefore, a distributed adaptive observer is introduced to learn the leader's dynamic model and estimate its state for each agent. For the tracking problem, an exponential discount value function is established and the related discrete-time game algebraic Riccati equation (DTGARE) is derived, which is the key to obtaining the control strategy. Furthermore, a data-based policy gradient algorithm is proposed to approximate the solution of the GAREs online and the utilization of agents' accurate knowledge is avoided. To improve the efficiency of data utilization, an offline dataset and the experience replay scheme are used. In addition, the lower bound of the exponential discount value is explored to ensure the stability of the systems. In the end, a simulation is provided to show the validity of the proposed method.