A Hybrid Deep Learning and Optimization Approach for Accurate Channel Estimation in 5G MIMO-OFDM Systems

Channel estimation plays a pivotal role in enhancing the reliability and efficiency of 5G wireless communication systems, particularly in MIMO-OFDM (Multiple Input Multiple Output - Orthogonal Frequency Division Multiplexing) architectures under multipath and Doppler-affected conditions. Conventional methods such as Least Squares (LS) are widely used due to their low computational complexity and lack of requirement for prior channel statistics. However, these approaches often result in poor estimation accuracy, especially in dynamic environments. To overcome these limitations, this study introduces a hybrid deep learning-based channel estimation framework that integrates Harris Hawks Optimization (HHO), Sparrow Search Algorithm (SSA), and Long Short-Term Memory (LSTM) networks—referred to as HHO-SSA-LSTM. The proposed method is designed to optimize the LSTM parameters using HHO and SSA, enhancing learning efficiency and estimation accuracy. Additionally, the model employs hybrid pre-coding aligned with codebook modeling strategies to preserve angle characteristics without disrupting azimuthal distributions. The system is evaluated in a 5G MIMO-OFDM setting under realistic conditions simulated using Doppler frequency and multipath propagation. Performance is assessed using key metrics including Bit Error Rate (BER), Mean Square Error (MSE), Symbol Error Rate (SER), efficiency, and execution time across different Pilot Lengths (PL = 128, 136, and 160). Simulation results demonstrate that the HHO-SSA-LSTM framework outperforms LS, LMMSE (Linear Minimum Mean Square Error), CNN (Convolutional Neural Network), FDNN (Forest Deep Neural Network), and standalone LSTM models. Notably, at PL = 160, BER is reduced by up to 91% and MSE by 86%, with an efficiency improvement exceeding 12% compared to traditional methods. Although the model exhibits a slightly higher execution time due to its hybrid design, the substantial accuracy gains justify the trade-off. The findings validate the effectiveness of the proposed hybrid model for robust and efficient channel estimation in 5G networks.