Designing reliable FSO links: selection of optimal dual-polarization modulation schemes based on turbulence conditions

Abstract Free-space optical (FSO) communication faces significant challenges owing to atmospheric turbulence, which can severely degrade the system performance. This paper presents a comprehensive performance analysis of various dual-polarization (DP) modulation schemes, including DP-differential phase shift keying (DP-DPSK), DP-differential quadrature phase shift keying (DP-DQPSK), DP-quadrature amplitude modulation with orthogonal frequency division multiplexing (DP-QAM-OFDM), and DP-phase-shift keying with OFDM (DP-PSK-OFDM), under turbulent FSO link conditions. Utilizing the OptiSystem simulation Software and the Gamma–Gamma turbulence model, the study evaluates these schemes’ bit error rate (BER) performance across varying link distances and turbulence intensities. The findings demonstrate a clear trade-off between spectral efficiency and robustness. DP-DPSK emerges as the most suitable option for long-range links, particularly under weak turbulence conditions, while DP-DQPSK demonstrates better performance in shorter-range scenarios with higher turbulence levels. Furthermore, higher-order DP-QAM-OFDM modulation exhibits increased sensitivity to turbulence, leading to significant performance degradation over extended distances. Conversely, lower-order DP-phase shift keying-OFDM modulation displays superior robustness, maintaining lower BERs even at longer ranges at the cost of reduced spectral efficiency. These findings provide critical guidance for selecting optimal modulation schemes in turbulence-affected FSO systems, highlighting the importance of balancing three key factors—turbulence resilience, spectral efficiency demands, and implementation complexity—to ensure reliable and high-performance optical links.