Improper Gaussian Signaling for Secrecy Transmission in RIS-Aided Downlink NOMA

This letter investigates the physical layer security of a downlink non-orthogonal multiple access (NOMA) system aided by a reconfigurable intelligent surface (RIS), where artificial noise (AN) is introduced to confuse the eavesdropper. To enhance secrecy, we employ improper Gaussian signaling (IGS) to alleviate the influence of AN on legitimate users. This is formulated as a sum secrecy rate maximization problem, where the reflecting coefficients of RIS and transmit covariance matrices of legitimate users and eavesdroppers are jointly optimized. An alternating optimization framework aided by the minorization-maximization method is adopted to address this non-convex optimization problem. Numerical results show that IGS significantly outperforms proper Gaussian signaling in improving the secrecy performance of RIS-aided NOMA systems, and such a secrecy advantage is more pronounced in RIS-aided NOMA than in traditional NOMA.