Discrete Softmax Policy Gradient for Statistical QoS Provisioning in RIS-Aided Holographic MIMO Networks

In this work, we investigate statistical quality-of-service (QoS) provisioning in a downlink reconfigurable intelligent surface (RIS)-aided holographic multiple-input multiple-output (MIMO) network. Holographic MIMO, enabled by nearly continuous aperture surfaces, offers fine-grained beamforming capabilities that can be leveraged for delay-sensitive and reliable communication. We aim to optimize discrete configurations of base station precoding, RIS phase shifts, and user-side combiners to meet statistical delay and reliability constraints. To this end, we propose a novel discrete softmax policy gradient (DSPG) framework that learns discrete control policies under effective capacity-based statistical QoS constraints. To enhance policy expressiveness and decision quality in large discrete action spaces, we integrate a generative diffusion model (GDM) into the agent architecture, enabling efficient sampling of high-quality configurations. Simulation results demonstrate that the proposed GDM-based DSPG significantly outperforms existing baselines, particularly under stringent latency and reliability requirements, making it a strong candidate for next-generation intelligent wireless systems.