Machine learning RNNs, SVM and NN Algorithm for Massive-MIMO-OTFS 6G Waveform with Rician and Rayleigh channel

Multiple Input and Output-Orthogonal Time–Frequency Selective (MIMO-OTFS) is considered one of the leading candidates for the beyond fifth generation (B5G) radio framework. The signal detection process is complex due to the large number of antennas, which also increases the framework’s latency. Signal detection algorithms such as Recurrent Neural Networks (RNNs), Neural Networks (NNs), Support Vector Machines (SVMs), Minimum Mean Square Error (MMSE), Maximum Likelihood Detection (MLD), Expectation-Maximization (EM), and Zero-Forcing Equalization (ZFE) are analyzed for Rayleigh and Rician channels. Currently available methods involve intricate identification and receivers with lower spectral efficiency. Experimental results indicate that RNNs, NNs, and SVM detectors, which have lower complexity, are recommended to improve the bit error rate (BER) and power spectral density (PSD) of the MIMO-OTFS system. It is also noted that RNNs offer diversity in received data, achieving a significant gain of 5 dB to 7 dB compared to existing OTFS systems across different MIMO frameworks. Furthermore, the utilization of machine learning algorithms significantly obtained a gain of −305 and −330 (RNNs) for the Rayleigh and Rician channels, respectively. These findings underscore the benefits of integrating sophisticated detection methods in B5G communication channels, indicating a valuable direction for future research and advancements in this area.