Recognizing Robotic Components in Complex Environment

Modern robotic applications demand precise visual perception to enable intelligent interaction with dynamic environments. In complex robotic working scenarios—such as multi-arm coordination in industrial automation, collaborative robotic assistants in healthcare, and humanoid robots interacting with humans—accurate robotic arm segmentation is critical for avoiding collisions, improving motion planning, and enhancing manipulation precision. Despite its importance, this challenge has been largely overlooked in the field, with no publicly available datasets specifically designed for robotic arm segmentation. To bridge this gap, we introduce the Robotic Arm Segmentation Dataset (RASD), a manually curated dataset comprising 3,269 images covering diverse robotic arms in various environments. Additionally, we propose a novel segmentation framework that refines the output of a pre-trained segmentation model by integrating spatial and semantic information. Our method employs a Large Vision Model (LVM) to enhance spatial awareness through a textual prompt: “Where is the robotic arm in the image?”. The extracted spatial and semantic features are then fused within a refined UNet, which takes an initial coarse segmentation mask as input and progressively refines it through a structured encoder-decoder process. Experimental results demonstrate that incorporating spatial-semantic fusion significantly improves segmentation accuracy compared to standard pre-trained models. By addressing a critical yet underexplored problem, our work contributes to safer and more intelligent robotic perception, paving the way for enhanced robotic autonomy in complex realworld scenarios.