Near-Field Hybrid Beamforming Design for Modular XL-MIMO ISAC Systems

A novel modular extremely large-scale multiple-input-multiple-output integrated sensing and communication system is investigated in this paper. The piecewise-far-field channel model is employed to characterize both communication and sensing channels, capturing the far-field propagation within each subarray and the near-field effects among subarrays due to the small subarray aperture and large inter-subarray spacing. Then, a joint transmit-receive beamforming problem is formulated to optimize communication spectral efficiency while satisfying the sensing signal-to-clutter-plus-noise ratio requirement. To solve this problem, an alternating optimization framework is proposed to iteratively update the transmit beamformer and receive beamformer until convergence. For a fixed receive beamformer, a closed-form optimal analog beamformer is firstly derived by exploiting the near-field propagation characteristics among subarrays, transforming the transmit hybrid beamforming problem into a low-dimensional digital beamforming optimization and substantially reducing the computational complexity. Then, two efficient algorithms are proposed to solve the rank-constrained digital beamforming problem. First, the semi-closed form of the optimal digital beamformer is derived and shown to form a complex Stiefel manifold. Based on this structure, a joint Riemannian-Euclidean gradient descent algorithm is developed for iterative optimization. Second, an semidefinite relaxation-based approach is proposed, where a near-optimal solution is obtained through rank constraint relaxation and randomization. Extensive simulations validate the superiority of the proposed algorithms, revealing that the optimal subarray scale balances spatial multiplexing and beamforming gains based on user distance, while increasing subarray numbers significantly enhances range resolution due to more pronounced spherical wavefronts.