A Transformer-Based Approach for Joint Interference Cancellation and Signal Detection in FTN-RIS MIMO Systems

Next-generation communication systems demand extreme spectral efficiency to handle ever-increasing data traffic. The combination of faster-than-Nyquist (FTN) signaling and reconfigurable intelligent surfaces (RISs) presents a promising solution to meet this demand. However, the aggressive time compression inherent to FTN signaling introduces severe and highly non-linear inter-symbol interference (ISI). This complex distortion is challenging for conventional linear equalizers and even for recurrent neural network (RNN)-based detectors, which can struggle to model long-range dependencies within the signal sequence. To overcome this limitation, this paper proposes a novel signal detection framework based on the transformer model. By leveraging its core multi-head self-attention mechanism, the transformer globally analyzes the entire received signal sequence at once. This enables it to effectively model and reverse complex ISI patterns by identifying the most significant interfering symbols, regardless of their position, leading to superior signal recovery. The simulation results validate the outstanding performance of the proposed approach. To achieve a target bit error rate (BER) of 10−4, the transformer-based detector shows a significant signal-to-noise ratio (SNR) gain of approximately 1.5 dB over a Bi-LSTM detector over 4 dB compared to the conventional FTN-RIS system, while maintaining a high spectral efficiency of nearly 2 bps/s/Hz.