A Novel RSMA-PIC Protocol for Massive MIMO Relaying Systems and its WSEE Optimization

We consider the downlink of a massive multiple-input-multiple-output (mMIMO) system, where a base station (BS) serves multiple clusters of single-antenna user equipments (UEs) by using multiple single-antenna relays. The BS employs a novel rate-splitting multiple access (RSMA) private interference cancellation (PIC) protocol wherein, similar to RSMA, a UE first decodes its common message. To decode its private message, a UE then cancels interference not only from the common message but, unlike RSMA, also from the private messages of weaker UEs in its own cluster. The RSMA-PIC protocol integrates the benefits of non-orthogonal multiple access (NOMA) and RSMA protocols, and outperforms them in low and high signal-to-noise regime. The BS employs minimum-mean-squared-error precoder to cancel the inter-relay interference between multiple relays. We also develop a decentralized two layer iterative optimization framework to maximize the weighted sum energy efficiency metric.