Leveraging Deep Reinforcement Learning for Spacecraft Configuration Design

Spacecraft configuration design, or the process of choosing the placement and orientation of each component in a spacecraft, is an essential aspect of the design process of satellites and can have a large impact on the eventual cost and performance of the spacecraft. Due to the complex and multi-domain nature of the problem, configuration design currently relies heavily on the intuition and experience of a team of engineers. The problem presented is, given a list of components and their characteristics (mass, dimensions, type, etc.), produce locations and orientations for each component which maximize performance while minimizing cost. To solve this, a deep rein-forcement learning (deep RL) implementation, Proximal Policy Optimization (PPO) is used. The deep RL implementation is benchmarked against a multi objective genetic algorithm (GA), specifically NSGA-II. It uses a gene size of four times the number of components (corresponding to the panel choice, x location, y location, and orientation of each component). The objective functions for the GA are the same as described for the deep RL method. Results show that the deep RL model while training performs as well as or slightly outperforms the GA, as measured by the hypervolume indicator. When paired with transfer learning, this method could allow for quicker and more thorough tradespace exploration in the early stages of spacecraft design. This method could in theory be adapted to other configuration design problems, such as chiplet design.