Development and Test of Deep Learning Techniques for Stereo-based Pose Estimation oo Small Satellites

For in-orbit servicing, spacecraft must conduct a variety of tasks known as Rendezvous & Proximity Operations, Docking (RPOD). The ability to accurately and reliably estimate a target object’s pose is required to perform these tasks. In computer vision, deep learning techniques been shown to significantly outperform classical techniques. However, there are several barriers to adoption of such technologies in spaceflight, related to development and verification/validation processes, and constraints of the target flight hardware. In this paper, we outline our approach and limited results from a recently completed project on the development and test of a neural network designed to allow a servicing spacecraft to estimate the pose of a target object in proximity, in real-time, using stereo vision, so as to enable planning for subsequent maneuvers. Developing and testing pose estimation approaches requires a reliable test bed capable of providing representativesimulation of relative movements as well as hardware analogous to that used for space applications. To address this, we first set up a facility we named the Orbital Autonomy Lab (OAL), which consists of two Universal Robots arms and an OptiTrack motion capture system. One arm was configured as a servicing spacecraft arm with a stereocamera attached; the other was used to place a mock satellite in several pose configurations, in view of the stereocamera. Using this setup, we collected 11,539 stereo pairs of a mock satellite in different positions and lighting configurations, emulating realistic approach and inspection paths. A core facet of this study was to gauge performance of neural networks on a flight-representative board. Given our previous experience in deploying CNNs for actual spaceflight missions, the Kria KV260 was selected as it is very similar to the popular Xiphos Q8 that was used in previous projects. For model development, we considered key constraints. First, the model must be lightweight to allow for quick inference on a flight-rated board. Second, the model needs to learn the target’s pose without prior knowledge of its shape. Third, the model needs to be generalizable across target poses and lighting conditions and approach scenarios. Following our previous experience in developing neural networks for spaceflight applications, we investigated many design approaches. During testing, we tracked L1 loss, orientation error, translation error, and inference speed, to gauge performance. Subsequently, we developed and tested three models, and selected one, dubbed PosenatorV2 with a dual MobileNetV3 backbone encoder as a final candidate for deployment on the KV260 for testing. Our targets for Mean Translation Error and Mean Angular Error were 0.5cm and 5° respectively. Through our final test results for the dual-arm sequences, we were able to achieve 0.61cm and 2.34° respectively. For assessing performance on the Kria KV260, we were able to achieve a time from image capture to inference at just under 1.5 seconds. In comparison, running this experiment with a model on a Heroku server, we were able to achieve a time of ~0.5 seconds.