Turbulence compensation based on pix2pixGAN for the free-space optical communication of orbital angular momentum multiplexing

Free space-optical communication utilizing orbital angular momentum (OAM) multiplexing offers an extremely high transmission capacity and speed. However, atmospheric turbulence causes mode crosstalk and power loss during the OAM decoding. Although conventional adaptive optics (AO) compensates the distortions, it introduces significant complexity and cost due to the need for a wavefront sensor. This paper proposes what we believe to be a novel sensor-less AO framework based on the image-to-image translation capability of pix-to-pix generative adversarial networks (pix2pixGAN). This approach directly recovers the compensation information by recognizing the distorted image. Laboratory experiments revealed that the proposed AO method effectively compensates the distortions of vortex beams in the 80 Gbit/s communication system combining four wavelengths and two OAMs. Even in severe turbulence (<i>D</i>/<i>r</i> ₀=8), the power sensitivity was minimally enhanced 3.7 dBm under the bit error rate of1∗10^-3.