Density-Based Hysteretic Learning for Fully Decentralised Environments

Independent agent policy learning has numerous applications in Multi-Agent Systems but faces challenges from the non-stationarity introduced by concurrent learning agents. Hysteretic learning is a common method for stabilising independent value-based policy learning. However, existing hysteretic learning methods’ reliance on the temporal difference error for weight assignment can emphasise sub-optimal behaviour. Our study addresses this problem by introducing novel clustering and ranking-based methods for determining hysteretic weights based on the target distribution density. Evaluations conducted on the PredatorPrey and cooperative matrix games show that the proposed methods outperform or can compete with baselines.