PPO-Based Energy Efficiency Maximization For RIS-Assisted Multi-User Miso Systems

In this paper, we explore the integration of a reconfigurable intelligent surface (RIS) with a multi-antenna base station (BS) for downlink multi-user multiple-input-single-output (MU-MISO) systems. We aim to enhance energy efficiency (EE) by jointly optimizing beamforming and phase shifts at the BS and RIS, respectively, while ensuring each mobile user meets their link budget requirements. The resulting optimization problem is inherently non-convex. To address this challenge, we employ proximal policy optimization (PPO), known for efficiently managing non-convex problems and reducing training overhead in continuous action spaces through a clip factor. Furthermore, by leveraging deep neural networks (DNN), the proposed PPO-based solution provides the optimum values for the beamforming at the BS and the phase shift at the RIS, respectively. Finally, we demonstrate the effectiveness and accuracy of the proposed PPO-based algorithm through an extensive simulation campaign, comparing its performance against baseline methods (i.e., fractional programming (FP) and deep deterministic policy gradient (DDPG)). The results show that our proposed PPO-based algorithm outperforms the considered baseline approaches (i.e., FP and DDPG) in terms of EE by 34.2% and 15.8%, respectively.