Performance-Driven End-to-End Optimization for UAV-Assisted Satellite Downlink with Hybrid NOMA/OMA Transmission

Unmanned aerial vehicle (UAV)-assisted satellite downlink transmission is a promising solution for improving coverage and throughput under challenging propagation conditions. However, the achievable performance gains are fundamentally constrained by the coupling between access transmission and the satellite–UAV backhaul, especially when decode-and-forward (DF) relaying and hybrid multiple access are employed. In this paper, we investigate the problem of end-to-end downlink sum-rate maximization in a UAV-assisted satellite network with hybrid non-orthogonal multiple access (NOMA)/orthogonal multiple access (OMA) transmission. We propose a performance-driven end-to-end optimization framework, in which UAV placement is optimized as an outer-layer control variable through an iterative procedure. For each candidate UAV position, a greedy transmission mode selection mechanism and a KKT-based satellite-to-UAV backhaul bandwidth allocation scheme are jointly executed in the inner layer to evaluate the resulting end-to-end downlink performance, whose feedback is then used to update the UAV position until convergence. Simulation results show that the proposed framework consistently outperforms benchmark schemes without requiring additional spectrum or transmit power. Under low satellite elevation angles, the proposed design improves system sum rate and spectral efficiency by approximately 25–35% compared with satellite-only NOMA transmission. In addition, the average user rate is increased by up to 37% under moderate network sizes, while maintaining stable relative gains as the number of users increases, confirming the effectiveness and scalability of the proposed approach.