Intelligent Optimization in Satellite Communication Protocols: Methods, Applications, and Practical Limits

Satellite communication protocols are increasingly optimized in software-defined, multiorbital networks that combine broadband satellite systems, non-terrestrial 5G components, and inter-satellite transport. This review examines intelligent optimization across the physical, medium-access, network, and transport layers, with emphasis on what can be measured, what can be controlled, and what can be safely deployed under standards and operational constraints. This paper first positions the literature across DVB/ETSI, 3GPP NTN, CCSDS/DTN, LEO routing, and recent AI and digital-twin research. It then links standards-defined control surfaces to layer-specific measurements, feedback delays, and safety constraints and compares optimization families using deployment-relevant criteria such as observability, runtime predictability, verification burden, and robustness. The review argues that the central challenge is not only a simulation-to-reality gap but an evidence gap between experimental gains and operational trust. To address this gap, this paper analyzes delayed observability, rare events, bounded onboard compute, action surface mismatch, certification, and security; formalizes a generic constrained optimization problem with delayed observations and standards-compliant actions; and proposes a digital-twin-assisted research methodology supported by a worked beam-hopping example. The main conclusion is that future progress is most likely to come from hybrid, standards-compliant, and twin-assisted optimization methods whose performance claims are tied to calibration, traceability, and explicit rollback logic.