Frequency-Domain Complex-Valued NN Equalizer Based on Deep Reinforcement Learning for Photonics-Assisted W-Band 16QAM-DMT Signals System Over 50 m

Photonics-assisted millimeter wave (MMW) can overcome the bandwidth bottleneck of electronic devices, but it also faces nonlinear damage caused by optoelectronic devices. Nonlinear equalization algorithms based on neural networks (NN) have gradually been introduced in the receiving end of 6G MMW communication, but the high computational complexity limits the real-time performance of the equalizer, and there is a tradeoff between the good equalization performance and low complexity. This paper proposes an Automatic Search Algorithm (ASA) based on deep reinforcement learning (DRL), which learns the functional relationship between equalizer structure parameters and optimization objectives through DRL, in order to quickly search for the optimal structure of various equalizers. The experiment demonstrated the W-band 16QAM discrete multi-tone (DMT) signal transmission in an intensity modulation and direct detection (IM/DD) system over a 50 m wireless link. We implement real-valued time-domain neural network equalizer (TDNNE), and complex-valued frequency-domain neural network equalizer (FDNNE), to solve the nonlinear issues respectively. Subsequently, ASA and greedy search algorithm were applied to TDNNE and FDNNE respectively, and the superiority of ASA was verified based on the bit error rate (BER) and the number of Multiply-Accumulate Operations (MACs) consumed by the equalizer. Finally, the experiment verified that using ASA to optimize the pruning of FDNNE can greatly reduce the complexity with almost no BER loss, while retaining the good equalization performance of NN equalizers and overcoming the disadvantage of high computational complexity. Thanks to our proposed complex-valued FDNNE based on DRL, the effective data rate reaches 11.4 Gb/s with 0.5 dB receiver sensitivity improvement compared to VNLE and TDNNE. At the same time, after pruning optimization, the average number of MACs consumed by FDNNE at various input optical powers into photodiode (PD) reduced by 37%. We believe ASA is an important application of using DRL to optimize neural networks in the field of nonlinear equalization. It can efficiently explore the performance limits of equalizers and has great potential for high-precision reception in future 6G wireless communications.