A Feedback-Based Distributed Method for Multiscale Optimal Control of Multi-Agent Systems

In this paper, we propose a distributed method for multiscale optimal control. Namely, we aim to optimize the macroscopic behavior of nonlinear multi-agent systems by acting on the microscopic dynamics of each agent using only local information. More specifically, we consider a finite-horizon optimal control problem in which the loss function is the sum of (i) local terms taking into account local inputs of each agent and (ii) a term depending on a macroscopic model of the network (e.g., the Kullback-Leibler divergence with respect to a target distribution). In particular, such a macroscopic model aggregates local kernels representing a probabilistic feature of a single agent (e.g., a local sensing model). The proposed method combines a local feedback-based action to optimize the macroscopic agents’ behavior with a learning part aimed at locally reconstructing the macroscopic model for each instant of the optimal control problem. We theoretically prove that the proposed method asymptotically converges to the set of stationary points of the optimal control problem. We achieve this result by exploiting tools from system theory based on timescale separation and LaSalle’s invariance principle. We test the proposed method with numerical simulations.