Channel-Awareness User Clustering and Adaptive Beamforming-Based Interference Mitigation Scheme in LEO-GEO Coexistence System

Low earth orbit (LEO) satellite communication systems have become the indispensable part of sixth generation (6 G) communications. However, since the LEO and geostationary earth orbit (GEO) satellite systems will inevitably share limited frequency resources, the communication signal from LEO satellites have the possibility to cause harmful interference to the GEO systems. To address this issue, this paper proposes an adaptive beamforming strategy to mitigate the interference while improving the system spectral efficiency (SE). In specific, we formulate the problem as the nonlinear mixed integer programming (NMIP) optimization problem, and apply the weighted minimum mean square error (WMMSE) and alternative optimization algorithm to obtain the closed-form solutions. Furthermore, to reduce the high complexity of beamforming when the LEO system serves a massive number of ground users (GU), we propose a channelaware user clustering scheme utilizing the channel correlation between GUs so that all GUs within the same cluster share the same precoding vector. Extensive simulations show that the proposed scheme effectively mitigates the interference.