Auto-Encoder Optimized PAM IM/DD Transceivers for Amplified Fiber Links

The use of semiconductor amplifier in integrated transceivers increases sensitivity, but changes the noise statistics in pulse amplitude modulation (PAM) intensity modulation with direct detection. Using a straight-forward, mixed noise model, we optimize constellations for these systems with an autoencoder-based neural network (NN). We improve required signal-to-noise ratio (SNR) by 4 dB for amplified spontaneous emission (ASE)-limited pulse amplitude modulation (PAM)4 and PAM8, without increasing system complexity. Performance can also be improved in O-band wavelength division multiplexing systems with semiconductor optical amplification and chromatic dispersion (CD). At 53 Gbaud, our simulations show we can extend the reach of PAM4 by 4 to 8 km when combining an optimized constellation with a NN decoder. We present an experimental validation of 4 dB improvement of an ASE-limited PAM4 back-to-back transmission at 60 Gbaud using an optimized constellation and a NN decoder.