DCDU-VMamba: Depth Completion of Transparent Objects With Dual U-VMamba for Robotic Grasp

Transparent objects are extensively employed in both industrial and medical field. However, transparent objects possess the intrinsic characteristics of specular reflection and refraction distortion, making it extremely challenging for robot to grasp due to the inability of capturing complete depth information via traditional machine vision approaches. To tackle this challenge, we propose the depth completion method of transparent objects with dual U-VMamba for robotic grasp (DCDU-VMamba), which is an end-to-end framework with RGB-D as the input. The proposed framework incorporates dual encoder subnetworks that integrate V-Mamba with U-Net, enabling the extraction of RGB and depth features across different semantic scales. Then through the powerful fusion network based on dual-head cross attention, the multi-scale RGB map and the depth map features are integrated. Finally, we further explore the contextual information during upsampling by using our self-designed decoder module. Comprehensive experiments show that our model provides a significant improvement compared to state-of-the-art approaches on the recently popular TransCG dataset. Our method demonstrates a 6% improvement in success rate when compared to the state-of-the-art approach in real-world robotic grasping experiments.