Deep Reinforcement Learning for Joint Observation and On-Orbit Computation Scheduling in Agile Satellite Constellations

Agile satellites leverage rapid and flexible maneuvering to image more targets per orbital cycle, which is essential for time-sensitive emergency operations, particularly disaster assessment. Correspondingly, the increasing observation data volumes necessitate the use of on-orbit computing to bypass storage and transmission limitations. However, coordinating precedence-dependent observation, computation, and downlink operations within limited time windows presents key challenges for agile satellite service optimization. Therefore, this paper proposes a deep reinforcement learning (DRL) approach to solve the joint observation and on-orbit computation scheduling (JOOCS) problem for agile satellite constellations. First, the infrastructure under study consists of observation satellites, a GEO satellite (dedicated to computing), ground stations, and communication links interconnecting them. Next, the JOOCS problem is described using mathematical formulations, and then a partially observable Markov decision process model is established with the objective of maximizing task completion profits. Finally, we design a joint scheduling decision algorithm based on multiagent proximal policy optimization (JS-MAPPO). Concerning the policy network of agents, a problem-specific encoder–decoder architecture is developed to improve the learning efficiency of JS-MAPPO. Simulation results show that JS-MAPPO surpasses the genetic algorithm and state-of-the-art DRL methods across various problem scales while incurring lower computational costs. Compared to random scheduling, JOOCS achieves up to 82.67% higher average task profit, demonstrating enhanced operational performance in agile satellite constellations.