Interpretable model-driven end-to-end deep learning of geometric shaping for DML-based IM/DD systems

Direct modulation lasers (DMLs) are a low-cost technology for short-reach intensity modulation and direct detection (IM/DD) systems due to their small footprint and low power consumption. However, their performance is limited by nonlinear distortion arising from chirp-dispersion interaction. Neural network (NN) autoencoders (AE)-based geometric shaping (GS) offers a promising solution through end-to-end (E2E) constellation optimization. This approach relies on differentiable and accurate channel models, leading to the adoption of NN-based models that require large training datasets and retraining for different configurations. In this Letter, we propose a low-complexity model-driven framework that establishes a surrogate channel based on composite second-order (CSO) distortion theory, enabling AE-based geometric constellation optimization to suppress chirp-dispersion interaction. This approach combines physics-based interpretability with deep learning's adaptive capabilities. Experimental results demonstrate a 1-dB receiver sensitivity improvement for 64-QAM signals after 10-km standard single-mode fiber (SSMF) transmission, confirming the effectiveness of the proposed scheme in mitigating nonlinear distortions.