Decision of Nonsynchronous Framework: Agents in MARL Have Different Priorities While Making Decisions

Multi-Agent Reinforcement Learning (MARL) faces key challenges in credit assignment and the curse of dimensionality as agent numbers grow. In cooperative settings, uniform treatment of agents often exacerbates these issues. We argue that an agent’s importance depends on its personalized attributes and environment states and propose concentrating computational resources on key agents while others act simply, alleviating dimensionality explosion and improving generalization. We propose the Decision of Nonsynchronous Framework (DNF), which identifies and prioritizes key agents at each time step for optimized decision-making, while assigning predefined or simplified behaviors to the remaining agents based on computational outcomes. To realize this, we introduce a Core Extractor (CE) architecture that categorizes agents into Priorities Key Agents (PKAs) and followers. Although agents are differentiated by priority, we still adhere to the Centralized Training with Decentralized Execution (CTDE) paradigm. This approach reduces the dimensionality of the joint state-action space, mitigates the dimensionality explosion problem in MARL, and fosters improved collaboration among agents. Experimental results demonstrate that DNF achieves a 100% win rate on multiple SMAC maps, including 3m, 2s3z, and 1c3s5z, and achieves 98.9–100% win rates on challenging hard and super-hard scenarios such as 2c_vs_64zg and Corridor, significantly outperforming baseline methods like QMIX and QPLEX in both final performance and training stability, while incurring only a modest increase in computational overhead. In the continuous MPE, DNF matches or exceeds HAPPO in performance and demonstrates substantially higher time efficiency, with both advantages growing more pronounced as the number of agents increases.