A Relative Orbital Motion-Guided Framework for Generating Multimodal Visual Data of Spacecraft

The advancement of on-orbit servicing and space debris removal missions has established high-precision visual perception for non-cooperative spacecraft as a key research focus. However, the availability of high-quality, diverse spacecraft image datasets is severely limited due to extreme on-orbit imaging conditions, data confidentiality, and morphological diversity of targets, significantly constraining the advancement of data-driven algorithms in this domain. To address this challenge, we propose a relative orbital motion-guided framework for generating multimodal visual data of spacecraft. The proposed method integrates an orbital dynamics model into the synthetic data generation pipeline to simulate typical relative motion patterns between the camera and the target in a realistic orbital environment, thereby generating image sequences characterized by continuous spatiotemporal evolution. Targeting four representative spacecraft—Tiangong, Spacedragon, ICESat, and Cassini—this work simultaneously generates a dataset comprising 8000 samples, each containing four strictly aligned modalities: RGB images, instance segmentation masks, depth maps, and surface normal maps, along with precise 6-degree-of-freedom (6-DoF) pose ground truth. Furthermore, an end-to-end physical image degradation model is developed to accurately simulate the complete imaging chain—from optical diffraction and aberrations to sensor sampling and noise—thereby effectively narrowing the domain gap between synthetic and real data. By addressing three key aspects—physical motion modeling, synchronous multimodal ground truth, and imaging degradation simulation—this work provides a crucial data foundation for training, testing, and validating data-driven on-orbit perception algorithms.