Bi-level graph attention paradigm with differential strategy integration for heterogeneous multi-agent reinforcement learning

Collaboration among heterogeneous agents is crucial for addressing complex real-world tasks that require leveraging diverse capabilities. In such systems, increasing agent numbers amplify the challenges of communication and coordinated decision-making, in addition to the inherent heterogeneity of the agents. To address these issues, we propose the Bi-level Graph Attention Paradigm (Bi-GAP) with differential strategy integration, a novel policy-based group learning framework designed for heterogeneous Multi-Agent Systems (MAS) in both discrete and continuous domains. Bi-GAP employs a bi-level graph attention architecture to model intricate interaction patterns among isomorphic agents within groups and across heterogeneous groups. This hierarchical representation enables flexible and selective communication, reduces unnecessary message exchange, and improves the robustness of the MAS under interference. Furthermore, the framework integrates multi-perspective strategies, allowing each member-agent to incorporate global guidance from its designated guide-agent while still performing fine-grained local reasoning. This mechanism balances macro-level coordination with micro-level adaptability. We evaluate Bi-GAP on heterogeneous StarCraft II micromanagement tasks and Multi-Agent Particle Environment Predator-Prey scenarios. The results show that Bi-GAP consistently outperforms recent state-of-the-art MARL baselines across both discrete and continuous settings.