Continual MARL-assisted Traffic-Adaptive GC-MAC Protocol for Task-Driven Directional MANETs

Faced with the abundance of mobile ad-hoc network (MANET) applications, there emerges a strong incentive to provision MANET in millimeter wave. However, the deafness of directional antennas and the decentralized structure of MANET make it difficult to achieve consistent medium access control (MAC) among nodes through random competition. Therefore, graph coloring-based MAC (GC-MAC) scheme is proposed to implement time division multiplexed scheduling, but it allocates equal slots to links, disregarding the unbalanced and piecewise stationary traffic distribution in task-driven MANETs. Here, we propose a continual multi-agent reinforcement learning (RL)-assisted traffic-adaptive scheme to enhance the agility of slot allocation. Specifically, we add a contention period to frames of GC-MAC, during which nodes analyze stochastic characteristics of traffic to derive link traffic distribution for each task, and adjust slot assignment to reach cooperation through decentralized multi-agent deep Q-network (MA-DQN). Besides, considering the traffic variation due to task switch, continual RL is incorporated to accommodate changes of the environment more sensitively. Finally, simulation results prove the proposed scheme achieves faster convergence speed, lower delay and higher throughput.