An Effective Approach for Spectral Efficiency Improvement in Massive <scp>MIMO</scp> Network Using Hybridized Optimization Assisted Optimal Pilot‐Based Vector Perturbation Precoding

ABSTRACT In multi‐user Massive Multiple Input Multiple Output (MIMO) systems, acquiring Channel State Information (CSI) at the transmission point is crucial for accurate estimation, but it fails by costs and complexity. The Massive MIMO networks are known for the improved Spectral Efficiency (SE). These systems are equipped with antenna groups at the receiving end and the transmission point. Analyzing Channel State Information (CSI) from faulty channels is maximizing the precoder's complexity. The complexity of determining the optimal disrupting vector improves power transmission but reduces SE. This makes the optimization process more challenging. Therefore, in this work, an Optimal Pilot‐Based Vector Perturbation Precoding (OPVP) is introduced to improve the SE of the massive MIMO system. The Hybrid Flamingo Search‐based Sparrow Search Optimization Algorithm (HFS‐SSOA) is used to optimally select the perturbing vector for efficient reception as well as transmission and is developed by combining the Flamingo Search Algorithm (FSA) and Sparrow Search Algorithm (SSA). In addition, the ideal pilot designs wisely intellects the CSI for providing response to the transmitter. Further, the compressive sensing will be used by OPVP for effectively selecting the low‐dimensional CSI. The suggested approach effectively detects the low dimensional CSI by considering the objective functions like computational complexity, transmitting power, and computational overhead which is used to develop the perturbing signal within the constellation bound. Finally, the simulation process is carried out on the developed model to prove its effectiveness.