Alternating Optimization of Phase Shift and Beamforming in a RIS-aided Hybrid NOMA System

This paper proposes a reconfigurable intelligent surface (RIS) aided hybrid non-orthogonal multiple access (NOMA) downlink transmission model. Specifically, the BS designs orthogonal beams based on the channels of the original nearby users. An edge target user is located within one of these beams, and a RIS is deployed in the other beam to provide an additional signal path for serving the edge user. An alternating optimization (AO) algorithm is proposed to jointly optimize the RIS phase shifts and beamforming coefficients, aiming to maximize the edge user's rate while ensuring the rate demands of the nearby users. Specifically, the problem is decomposed into two subproblems: 1) optimizing RIS phase shifts via semidefinite relaxation (SDR) with fixed beamforming, and 2) deriving closed-form beamforming solutions with fixed RIS phase shifts. Simulation results show that the proposed AO algorithm achieves a notable rate improvement compared to the random-phase scheme and the non-RIS scheme. Additionally, the impact of the number of base station antennas and RIS elements is analyzed.