Performance Analysis of a Low-Complexity Hybrid DF–SIR Protocol for Coded Cooperative Hybrid Satellite–Terrestrial Networks

Abstract Hybrid satellite-terrestrial cooperative networks (HSTCNs) are a key enabler of sixth-generation (6G) non-terrestrial networks, providing resilient wide-area coverage under heterogeneous propagation conditions. This paper proposes a low-complexity hybrid decode-and-forward and soft-information-relaying (HDF-SIR) protocol for coded cooperative HSTCNs operating over mixed satellite-terrestrial fading channels. In the proposed scheme, each relay adaptively switches between decode-and-forward (DF) and soft-information-relaying (SIR) modes according to its decoding reliability, allowing unreliable relays to forward reliability-weighted soft symbols instead of being excluded from cooperation. The satellite-relay and satellite-destination links are modeled by shadowed-Rician fading, while the relay-destination hop follows a Nakagami-m distribution, capturing both satellite shadowing effects and terrestrial channel variability. To enable analytical tractability, a moment-based statistical model is developed for the soft symbols generated at the output of the relay’s soft-input soft-output decoder, leading to an equivalent soft-channel representation. Based on this model, closed-form and semi-analytical expressions are derived for the end-to-end signal-to-noise ratio (SNR), outage probability, and symbol error rate (SER) with opportunistic relay selection. Monte Carlo simulations validate the analysis and demonstrate that the proposed hybrid DF-SIR protocol preserves cooperative diversity while significantly improving reliability compared with conventional DF-only and SIR-only relaying, particularly in the mid-SNR regime where decoding reliability is uncertain. Owing to its direct reuse of decoder-generated log-likelihood ratios without additional re-encoding or iterative processing, the proposed scheme achieves a favorable performance-complexity trade-off, making 1 it a practical and energy-efficient solution for future 6G hybrid satellite-terrestrial communication systems.