Emerging Generalized Wireless MAC Communication Protocols via Abstraction

In this work, we propose an automated wireless medium access control (MAC) protocol design based on multi-agent reinforcement learning (MARL), in which multiple wireless devices (WDs) and a base station (BS) exchange control messages without prior knowledge of their meanings to coordinate data delivery across the network. In current approaches, the BS acts as a MAC expert, while the WDs are the only learning entities. As a result, the learned MAC protocols are constrained by the BS’s predefined knowledge. In contrast, our approach enables both WDs and the BS to act as learning agents, fostering the collaborative development of new ad hoc communication protocols. However, this approach introduces additional challenges in managing heterogeneous agents that learn simultaneously, alongside conventional issues such as the learned protocols being overfitted to training parameters, including the number of transmitted packets and the number of WDs managed by the BS. To overcome these limitations, we propose a novel learning framework that facilitates the emergence of MAC protocols with robust generalization capabilities. This framework incorporates innovative features such as a novel BS policy model, two distinct reward functions for the WDs and the BS, and different update routines for the agents’ policies. Moreover, we leverage the concepts of parameter sharing and state abstraction to enhance generalization in the WDs’ policies. The MAC protocols generated using the proposed framework are evaluated against state-of-the-art approaches. Simulation results show that the proposed solution significantly outperforms the benchmarks in terms of generalization capabilities.