MAIDTB: Multi-Agent Intelligent Dynamic Trajectory Backtrack for Sparse Strategic Games

Multi-agent reinforcement learning (MARL) in sparse strategic games, characterized by intelligent dynamic targets and infrequent feedback, poses significant learning challenges. This paper introduces the Multi-agent Intelligent Dynamic Trajectory Backtrack (MAIDTB) framework, a novel method to augment MARL algorithms for these complex scenarios. MAIDTB employs Bayesian inverse models to predict strategic target behaviors, enabling the generation of valuable ‘imagined’ trajectories from failed attempts. A Bayesian Neural Network (BNN) is integral to this process, assessing the confidence of inverse model predictions to dynamically control trajectory inference step length. This approach enhances learning robustness by mitigating error accumulation from uncertain predictions. Comparative experiments demonstrate that MARL agents augmented with MAIDTB achieve significantly higher learning efficiency and task success rates in sparse strategic games compared to both baseline MARL algorithms and those augmented with other existing trajectory replay techniques.