CoVarNet: Contrastive variational network for communication in multi-agent reinforcement learning

In multi-agent reinforcement learning, communication plays an important role in cooperation, yet most existing approaches neglect proper modeling of communication channel noise. In most scenarios, additive white Gaussian noise with dynamic signal-to-noise ratios is unavoidable. Such noise will interfere with the messages, preventing the receiver from accurately understanding the sender’s intentions. This phenomenon will affect the decision-making of the agent and ultimately result in the failure of the collaborative task. To address this, we propose Contrastive Variational Network (CoVarNet), a novel framework for robust cooperative communication. CoVarNet leverages a variational autoencoder to extract latent representations from noisy messages, and deploys contrastive learning to enhance the noise invariance of these extracted features. By perturbing the message vector and maximizing the similarity between the message and the perturbed one, while distinguishing it from messages from different episodes, agents learn to recover the underlying tactical intent despite channel corruption. We evaluate the proposed method on various standard benchmarks. The results indicate that CoVarNet perform better than baseline algorithms.