Genetic algorithm-based PTS with CNN for PAPR and BER reduction in FBMC systems under fading channels

The Filter Bank Multicarrier (FBMC) is a promising candidate for next-generation wireless systems because of its superior spectral efficiency and resilience to synchronization errors. However, its high Peak-to-Average Power Ratio (PAPR) remains a critical challenge that affects the power efficiency and nonlinear distortion performance. This study proposes a GA-assisted partial transmit sequence with a convolutional neural network (GA-PTS + CNN) technique to effectively mitigate PAPR in FBMC systems. The method optimally selects phase factors using a Genetic Algorithm (GA) while leveraging a CNN for adaptive learning, accelerating convergence, and improving system robustness. The proposed approach was validated through numerical simulations under Rayleigh and Rician fading channels and compared with conventional PAPR reduction techniques, including Clipping and Filtering (C&F), Selective Mapping (SLM), Partial Transmit Sequence (PTS), and Particle Swarm Optimization (PSO)-aided PTS. The results demonstrate a 2–3 dB PAPR reduction, lowering the FBMC’s peak PAPR from 10 dB to approximately 7 dB in Rayleigh fading, with a 1.5–3 dB reduction in Rician fading. Bit Error Rate (BER) analysis reveals an SNR gain of 2.5–6.5 dB at BER = 10 −4 with 256-QAM in Rayleigh fading and 3–5 dB gain in Rician fading, with consistent improvements for 64-QAM (4–10 dB gain). Additionally, Power Spectral Density (PSD) analysis confirmed the enhanced spectral efficiency. These findings highlight GA-PTS + CNN as a practical and robust solution for future wireless networks, including 5G and beyond, in which PAPR reduction, BER improvement, and efficient spectrum utilization are crucial.