SynBag: Synthetic Training Data for Autonomous Grasping of Regolith Bags in the Lunar Environment

Accurate perception of deformable objects on the lunar surface is essential for autonomous construction and in situ resource utilization (ISRU) missions. However, the lack of representative lunar imagery limits the development of data-driven models for pose estimation and manipulation. We present SynBag 1.0, a large-scale synthetic dataset designed for training and benchmarking six-degree-of-freedom (6-DoF) pose estimation algorithms on regolith-filled construction bags. SynBag 1.0 employs rigid-body representations of bag meshes designed to approximate deformable structures with varied levels of feature richness. The dataset was generated using a novel framework titled MoonBot Studio, built in Unreal Engine 5 with physically consistent lunar lighting, low-gravity dynamics, and dynamic dust occlusion modeled through Niagara particle systems. SynBag 1.0 contains approximately 180,000 labeled samples, including RGB images, dense depth maps, instance segmentation masks, and ground-truth 6-DoF poses in a near-BOP format. To verify dataset usability and annotation consistency, we perform zero-shot 6-DoF pose estimation using a state-of-the-art model on a representative subset of the dataset. Variations span solar azimuth, camera height, elevation, dust state, surface degradation, occlusion level, and terrain type. SynBag 1.0 establishes one of the first open, physically grounded datasets for 6-DoF-object perception in lunar construction and provides a scalable basis for future datasets incorporating soft-body simulation and robotic grasping.