GoMIC: Enhancing Efficient Collaboration in Multiagent Reinforcement Learning Through Group-Specific Mutual Information

In cooperative multi-agent reinforcement learning (MARL), previous research has predominantly concentrated on augmenting cooperation through the optimization of global behavioral correlations between agents, with mutual information (MI) typically serving as a crucial metric for correlation quantification. The existing approaches aim to enhance the behavioral correlation among agents to foster better cooperation and goal alignment by leveraging MI. However, it has been demonstrated that the cooperative capabilities among agents cannot be enhanced merely by directly increasing their overall behavioral correlations, particularly in environments with multiple subtasks or scenarios requiring dynamic team structures. To tackle this challenge, a MARL algorithm named group-oriented mutual information collaboration (GoMIC) is designed, which dynamically partitions agents and employs MI within each partition as an enhanced reward. GoMIC mitigates excessive reliance of individual policies on team-related information and fosters agents to acquire policies across varying team compositions. Experimental evaluations across various tasks in multi-agent particle environment (MPE), level-based foraging (LBF), and StarCraft II demonstrate the superior performance of GoMIC over some existing approaches, indicating its potential to improve collaboration in multi-agent systems.