A hybrid transformer–zero-shot learning framework with Muon optimization for intelligent channel estimation in MIMO wireless systems

For MIMO wireless systems, accurate channel estimation is essential. However, traditional and current Deep Learning (DL) techniques have poor generalization to unknown situations and necessitate repeated retraining. For intelligent MIMO channel estimation, this study suggests a novel hybrid framework that combines Transformer designs, Zero-Shot Learning (ZSL), and the Muon optimizer. By projecting channel instances into a shared semantic-attribute space, the ZSL component allows precise inference under previously unknown SNR levels and fading conditions without retraining. The Muon optimizer offers better generalization and faster convergence than Adam, while the Transformer uses self-attention to capture intricate spatial-temporal connections. ZSL-Muon, Transformer-ZSL, and the whole Transformer-ZSL-Muon model are the three configurations that are assessed. The whole hybrid consistently outperforms LS, MMSE, CNN-, GRU-, and Transformer-only baselines in MSE over a broad SNR range, according to extensive simulations under quasi-static and time-varying fading. The proposed framework is rigorously evaluated over a wide SNR range and under both quasi-static block fading and time-varying fading channel models. Experimental results demonstrate that the hybrid Transformer-ZSL-Muon model consistently surpasses traditional estimators (e.g., LS and MMSE) and existing DL-based techniques in terms of Mean Squared Error (MSE), particularly in complex and unseen channel conditions. A remarkable finding reveals that increasing the number of antennas from 2 to 128 at an SNR of 30 dB results in a 94.44% MSE reduction, underscoring the potential of massive MIMO implementations. Furthermore, the proposed framework achieves approximately 93.75% MSE improvement over MMSE and 96.67% over LS under the same SNR conditions. These outcomes affirm the effectiveness of combining semantic-aware inference, attention-driven Transformer architectures, and adaptive optimization to deliver a scalable, intelligent, and resilient solution for next-generation MIMO wireless communication systems.