Excellent-student learning method for decentralized MARL with networked agents system

Abstract Multi-agent reinforcement learning has been widely applied in solving various sequential decision-making problems in recent years. However, a key challenge is sparse rewards, where agents receive meaningful reward signals only upon task completion. This issue leads to inefficient exploration and slow learning progress. To address this problem, we propose an excellent-student learning method supported by a decentralized distributed learning paradigm, drawing inspiration from academically diverse classrooms. Specifically, we design a novel excellent-student learning model, which suggests that agents mimic the learning behaviors of excellent students. This model requires agents to share knowledge with other agents and engage in individual exploration driven by curiosity. Next, to foster collaborative team learning behaviors, similarity measurement techniques are integrated to enhance knowledge sharing among agents. An intrinsic reward function is designed, combining individual exploration with whole-class sharing, providing additional motivation for discovering new actions and states. This reward function is seamlessly incorporated into the policy learning process. Finally, experiments conducted in various multi-agent particle environments demonstrate significant improvements in training efficiency and stability.