A Feature-Enhanced Deep Learning-Based Signal Detection Algorithm for MU-MIMO-NOMA Systems

Deep learning (DL)-based signal detection al-gorithms for multi-user multiple-input multiple-output non-orthogonal multiple access (MU-MIMO-NOMA) systems can avoid the error propagation caused by iterative interference cancellation compared to operations research based methods but are still limited by high computational complexity. To address the aforementioned challenge and improve the re-training speed when the modulation mode changes, a data-driven feature-enhanced deep learning (DL)-based hierarchical transfer learning-augmented (DD-DTL) signal detection algorithm with low complexity is proposed, which introduce a nonlinear transformation layer to extract the high-dimensional features of receive signals. The proposed algorithm integrates the channel estimation and signal detection, performing implicit successive interference cancellation (SIC) to avoid the error propagation. Additional, a hierarchical transfer learning strategy is combined to improve the retraining speed under different modulation schemes, enabling the network to quickly update with only 5% of the training dataset while achieving good signal detection performance.