Design and Optimization of Intelligent FSO System Using DSP and Adaptive Modulation for Next Generation Cellular Networks

Free-space optics (FSO) is being considered for 5G and next-generation wireless networks for its free spectrum, short response time, high data rates, and innate security. We designed a dual-polarization FSO system with adaptive modulation at the receiver and advanced digital signal processing (DSP) to mitigate atmospheric turbulence and polarization impairments. Simulation results demonstrate that adaptive modulation significantly improves bit error rate (BER) and spectral efficiency under varying turbulence levels. A full DSP algorithm, including filtering, equalization, and carrier recovery, reduces BER by orders of magnitude in severe conditions. We analyze system performance under various weather scenarios (clear air, haze, fog) and polarization-related impairments (polarization-dependent loss, crosstalk, and rotation). The proposed dual-polarization FSO system achieves high data rates (≥70 Gbps) over extended distances (∼10 km) while maintaining acceptable BER. These results provide valuable insights for integrating FSO links into 5G and future networks, highlighting the benefits of polarization multiplexing, adaptive techniques, and DSP in overcoming FSO channel challenges. The findings will aid network planners in deploying robust high-capacity FSO links for next-generation cellular networks.