Machine Learning Assisted Performance Analysis of RoFSO System with Different Modulation Techniques

The rapid and continuous growth in demand for advanced communication system applications has led to the emergence of numerous new research directions aimed at addressing critical challenges related to speed, bandwidth efficiency, reliability, and security. In response to these demands, particular emphasis has been placed on the modernization of optical communication systems and the adaptation of modulation techniques to meet evolving requirements. This study investigates the fundamental modulation schemes commonly employed in optical communication systems, specifically quadrature amplitude modulation (QAM) and phase-shift keying (PSK). Using optical simulation software, a radio-over-free space optical (RoFSO) communication system is designed and its performance is analyzed. Furthermore, the simulation data were utilized in a Python™ environment to develop predictive models using several machine learning (ML) techniques: decision tree (DT), random forest (RF), support vector regression (SVR), multiple linear regression (MLR), and artificial neural networks (ANN). The performance of each ML model was assessed based on prediction accuracy. The main purpose of this paper is to develop a hybrid communication system that provides an efficient transmission environment by comparing different modulation techniques over the RoFSO link and also to build ML models that can predict system performance metrics with high accuracy using different ML methods. The results demonstrate that lower-order modulation schemes, such as 4-QAM and QPSK, yield superior performance compared to higher-order alternatives. Moreover, PSK-based modulation was found to outperform QAM in terms of overall system efficiency. Among the five ML models evaluated, prediction accuracies ranged from 74.9% to 93.2%, with an average accuracy of 84.5%. The random forest model achieved the highest performance, attaining a prediction accuracy of 93.2%.