Swarm Imitation Learning From Observations

Learning from observation (LfO) is a process where an agent learns a task by passively observing a more competent agent perform it. LfO differs from classical Learning from demonstration (LfD) in that the former requires access to the demonstrator' s states only, whereas the latter requires both the demonstrator' s states and the corresponding actions. On the one hand, LfO avoids the sometimes costly or impractical burden of collecting the demonstrator' s actions, and instead only requires the demonstrator' s states which are more easily captured through cameras or sensors. On the other hand, LfO is more challenging than classical LfD because of the lack of detailed guidance from action labels. Despite the success of LfO in single-agent tasks, the literature falls short of assessing its feasibility in swarm systems, where multiple agents act simultaneously to enact a system-level state change. We tackle this research gap by proposing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swarm-LfO</i> that extends single-agent LfO by leveraging the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">centralised training with decentralised execution</i> framework to learn a useful agent-centric inverse dynamic model (AIDM). AIDM enables the imitator swarm to predict agent-level actions that would lead to swarm state transitions similar to those exhibited by the demonstrator swarm. Pairs of states and the corresponding estimated actions are then used for learning to imitate the demonstrated behaviour in a supervised learning manner. Evaluation experiments are conducted using four tasks that require different levels of coordination between swarm members: flocking, sheltering, dispersion, and herding. The results show that the performance of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swarm-LfO</i> is comparable to classical LfD methods that require access to action information. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swarm-LfO</i> is extensively evaluated and has demonstrated continued success under various experimental conditions including noise and different sizes of the demonstrator and imitator swarms. Our contribution will pave the way for imitation learning in swarms with diverse platforms, where the demonstrator and imitator swarms operate on different action spaces.