An Automatically Annotated Spacecraft Intelligent Perception Dataset Based on Segment Anything Model

Spacecraft intelligent perception, including pose estimation and target detection, plays a vital role in the relative navigation system for spacecraft rendezvous and docking and active debris removal. Deep learning-based methods are widely used among spacecraft relative state perception. However, labeled images of on-orbit spacecraft for model training are difficult to obtain. In this paper, we propose the Spacecraft Object Detection and Pose Estimation Dataset (SDPED), generated by Unreal Engine 4 (UE4) and hardware in the laboratory (HIL), respectively. Additionally, we utilize the segment anything model (SAM) to autonomously annotate UE4 images through small-batch training. The advantages of the proposed SDPED are that it includes a variety of spacecraft types, a variety of label information, and an autonomous and accurate labeling method. Subsequently, we modified the multi-task network PVSPE by retaining only the pose estimation and target detection heads to evaluate the effectiveness of the SDPED. Extensive experiments are conducted on challenging synthetic and hardware-in-the-loop images. The results demonstrate that average estimation errors of position and attitude, as well as average IoU, on synthetic images, are 0.55, 6.11°, and 0.91, respectively. Moreover, the model generalizes well to HIL images through data augmentation and self-attention mechanism.