Learning effective nonlinear operator for high speed optical compensation system

Optical communication systems, which form the backbone of modern communication networks, are often hindered by impairments arising from both linear and nonlinear Kerr effects during signal transmission. While Chromatic Dispersion Compensation (CDC) mitigates linear distortions, existing digital signal processing (DSP) algorithms focus on addressing nonlinear Kerr effects. Current compensation architectures, including post-processing compensation (PPC) and digital back-propagation (DBP), each have limitations: PPC's separate handling of linear and nonlinear effects reduces compensation accuracy, while DBP's iterative nature incurs high computational complexity. Neither PPC nor DBP achieves efficient nonlinear compensation. This motivates the need for a method that unifies the strengths of both algorithms and optimizes the trade-off between performance and computational cost. In this paper, we present SNSE, the simplified nonlinear symbol equation, as our theoretical contribution, which provides a unified framework for the design of nonlinear compensation operators. Based on SNSE, we introduce what we believe to be a compensation framework, SNSE-DBP. SNSE-DBP leverages a regular cross-shaped truncation method for nonlinear effects, enabling efficient convolutional modeling for the nonlinear compensation operator. Furthermore, we translate SNSE-DBP into a deep neural network to optimize performance via data-driven training. Numerical simulations demonstrate that SNSE-DBP outperforms both the PPC scheme and DBP scheme in efficiency. This work presents a promising pathway toward effective and computationally feasible nonlinear compensation in optical communication systems.