High-speed channel equalization based on a hybrid neural network architecture

In digital communication, data transmission is subject to distortion caused by physical characteristics such as channel multipath propagation and Doppler frequency shift, as well as noise interference. Among these, inter-symbol interference (ISI) induced by channel dispersion leads to energy overlap between adjacent symbols, severely limiting the reliability and rate of transmission. To enhance the reliability of digital communication and support high-speed transmission, it is essential to overcome the performance limitations of traditional equalization techniques in time-varying multipath channels and effectively suppress distortions such as intersymbol interference and Doppler frequency shift. This paper proposes a hybrid equalizer architecture combining a CNN-Transformer-GRU framework with a cyclic shift preprocessing technique. This model generates an 18-dimensional feature matrix by performing nine groups of cyclic shifts on the received signal, transforming the time-varying channel features into a spatial dimension representation. By combining CNN local feature extraction, Transformer global dependency modelling, and GRU dynamic temporal adjustment, efficient channel equalization is achieved. Experimental results carried out under the SUI-3 channel model indicate that, on the premise of the same bit error rate (BER), the proposed model achieves a signal-to-noise ratio (SNR) gain of approximately 1–8 dB and 1–6 dB in comparison with the traditional decision feedback equalizer (DFE) and the temporal model LSTM, respectively, and improves the performance by 1–3 dB compared with the CRNN-RES (Convolutional recurrent neural network with ResNet) model, while maintaining a robustness of 2–8 dB. The parallel computation of transformers improves training efficiency by 30%, while the GRU architecture reduces the number of parameters by 20%. This architecture provides a high-precision, low-complexity solution for high-speed dynamic channel equalization in 5 G/6G systems.