Grey-Box Adversarial Attack on Communication in Communicative Multi-Agent Reinforcement Learning

Effective communication is a necessary condition for intelligent agents to collaborate in multi-agent environments. Although increasing attention has been paid to communicative multi-agent reinforcement learning (CMARL), the vulnerability of the communication mechanism in CMARL has not been well investigated, especially when there exist malicious agents that send adversarial communication messages to other regular agents. Existing works about adversarial communication in CMARL focus on black-box attack where the attacker cannot access any model within the multi-agent system (MAS). However, a more practical attack is grey-box attack, where the attacker has access to the models of their controlled agents. To the best of our knowledge, no research has been conducted to investigate grey-box attacks on communication in CMARL. In this paper, we propose the first grey-box attack method on communication in CMARL, which is called victim-simulation based adversarial attack (VSAA). At each timestep, the attacker simulates a victim attacked by other regular agents’ communication messages and generates adversarial perturbations on its received communication messages. The attacker then sends the aggregation of these perturbations to the regular agents through communication messages, which will induce non-optimal actions of the regular agents and subsequently degrade the performance of the MAS. Experimental results on multiple tasks show that VSAA can effectively degrade the performance of the MAS. The findings in this paper will make researchers be aware of the grey-box attack in CMARL.