Route Optimization in Satellite-to-Satellite Optical Communication Networks with Deep Reinforcement Learning

In a multihop communication using low earth orbit(LEO) satellite networks between two ground stations, latency increases when access to satellites along the route is congested. In this paper, we propose a route selection method using a deep reinforcement learning algorithm to obtain a communication path with reduced delay. For each satellite, deep reinforcement learning is performed using the states of neighboring satellites to select the next hop satellite for relaying. We employ the Deep Q-Network as a reinforcement algorithm, with the queuing delay and distance to the destination ground station used as state information. The reward is defined as the difference in latency between the shortest distance route and the route selected by the algorithm. We evaluate the performance of the proposed method through computer simulations of dynamic LEO satellite constellations. The results demonstrate that the proposed method effectively avoids satellites experiencing high traffic and selects routes with shorter delays than the shortest distance route.