Meta-Reinforcement Learning for Emergent Multi-Agent Languages in Zero-Shot Coordination Tasks

Recently, emergent communication protocols among agents have been increasingly applied to solve complex multiagent coordination tasks. However, most current approaches lack the ability to adapt quickly and efficiently to novel tasks and adversarial conditions without retraining. This paper introduces a new framework that integrates meta-reinforcement learning (meta-RL) with hierarchical reinforcement learning (HRL) to enable the development of emergent communication protocols by agents, which turn out to be robust, compositional, and adapt in a zero-shot manner to unseen tasks and perturbations. We concretely propose a meta-learning scheme that learns the prior over communication from a diverse set of training scenarios. These learned priors are used by agents at test time to rapidly infer new protocols suitable for unseen tasks or adversarial interference in communication channels. We perform detailed synthetic experiments on a suite of benchmark coordination problems under various adversarial conditions. Results indicate that our approach outperforms baseline methods in terms of zero-shot adaptation performance, resilience to noise, and crossdomain scalability. Our work opens up avenues for scalable, robust, and adaptive communication strategies in multi-agent systems, with wide-ranging implications for autonomous fleets, Internet of Things (IoT) networks, and beyond.