Macro-Action-Based DRL for Asynchronous Multi-User Dynamic Spectrum Access in Half-Duplex CRNs

This paper addresses the asynchronous multi-user dynamic spectrum access (DSA) problem in half-duplex cognitive radio networks (HD-CRNs). The core challenge lies in resolving the decision asynchrony caused by variable action duration under HD constraints, which traditional deep reinforcement learning (DRL) methods fail to address. The problem is formulated as a Macro-Action Decentralized Partially Observable Markov Decision Process (MacDec-POMDP), with a decentralized macro-action-based multi-agent Dueling Double Deep Q-Network (D3QN) framework being proposed. An extra reward mechanism is specifically designed to encourage sustained conflict-free access. Simulation results demonstrate that our approach outperforms synchronous fixed-duration methods by dynamically adjusting access duration. The reward function design significantly improves adaptability, reducing collisions to both primary and secondary users while optimizing the spectrum utilization under HD constraints, validating its effectiveness in asynchronous multi-user DSA.