Throughput maximization using reinforcement learning with state aggregation in satellite revisit scenarios

Abstract Low Earth Orbit (LEO) satellites are increasingly requiring higher data transfer rates to accommodate the volume of data generated by onboard instruments, such as hyperspectral cameras and synthetic aperture radars. This work proposes a reinforcement learning–based solution to maximize throughput. The central idea is to define a reward function that jointly accounts for packet error rate and modulation scheme. Using the Q-learning algorithm, the agent learns to dynamically select the appropriate modulation. To address the challenge of an infinite state space, an efficient state aggregation strategy is introduced. The evaluation scenario is constructed using real orbital data from the Argentine SAOCOM-1B satellite, represented by Two-Line Elements (TLEs), with a ground station located in Córdoba, Argentina. The channel model assumes free-space path loss (FSPL) and a steerable antenna characterized by its radiation pattern. Four modulation schemes are considered: QPSK, 8-PSK, 16-QAM, and 32-QAM. Numerical results demonstrate that the proposed RL algorithm successfully learns both antenna steering actions and modulation selection, thereby maximizing throughput.