TD3-Based Visual-Tactile Fusion for Dexterous Robotic Grasping

Robotic grasping in unstructured environments presents significant challenges due to complex object geometries, varying material properties, and dynamic operational conditions. While conventional vision-based grasping systems offer spatial awareness, they often fail to address critical contact dynamics during manipulation. Conversely, tactile-based approaches, though sensitive to contact interactions, lack global situational understanding. Therefore, this paper presents a novel visual-tactile fusion framework based on twin delayed deep deterministic policy gradient framework that leverages the synergistic advantages of visual and tactile modalities to improve autonomous learning and control capabilities for humanoid robotic hands in unstructured environments. Simulation results demonstrate that the proposed framework achieves rapid convergence during training, maintaining an 85% grasping success rate, which is much higher than the 50% grasping success rate based on the proximal policy optimization (PPO) framework, proving that the proposed framework has better grasping performance. During testing, it attains high success rates of 94.55% on training objects and 89.63% on previously unseen objects, significantly outperforming the PPO method's performance of 52.78% and 45.83%, confirming its superior adaptability and generalization in unstructured settings. The proposed method effectively combines multimodal perceptual information to enhance grasping performance while improving operational adaptability and generalization capability. This work provides new insights for intelligent robotic grasping research in unstructured environments.