Modeling and Machine Learning Assisted Optimization of <scp>FSS</scp> Integrated Tree‐Shaped <scp>MIMO</scp> Fractal Antenna With Enhanced Gain for Next‐Generation <scp>5G</scp>  mm‐Wave Communication Systems

ABSTRACT This research presents the design and optimization of a miniaturized frequency‐selective surface (FSS)‐embedded fractal MIMO antenna for 28 GHz 5G New Radio (NR) applications. The compact antenna, measuring 4.2 × 1.87 λ, employs a 7 × 3 unit‐cell array to achieve superior radiation characteristics. The design methodology integrates fractal geometry analysis, FSS loading for electromagnetic enhancement, and machine learning‐based optimization to achieve optimal gain, impedance bandwidth, and radiation efficiency. The antenna, realized on a Rogers RT5880 substrate, is optimized using Random Forest (RF), Artificial Neural Network (ANN), and regression‐based predictive models. Performance evaluation using R 2 , MAE, and MSE confirms the robustness of the optimization framework. Key design challenges include maintaining high isolation among compact MIMO elements, ensuring stable broadband response, and balancing miniaturization with gain without sacrificing efficiency. These challenges are addressed through precise geometric refinement, effective FSS coupling, and iterative machine learning tuning. Experimental validation using a ZVA 67 vector network analyzer demonstrates a broad impedance bandwidth of 2200 MHz (26–28.2 GHz), a peak gain of 7.53 dBi, and a radiation efficiency of 88.49%. The FSS integration enhances isolation, suppresses surface waves, and improves overall mm Wave performance. The proposed ML‐assisted FSS‐MIMO design provides an efficient solution for next‐generation 5G wireless communication systems.