Efficient Slip Detection in Robotic Grasping Using Accelerated TranSformers and Multimodal Sensor Fusion

We present introduces VTF - AVIT, an innovative TranSformer- based model that integrates tactile and visual sensors to enhance slip detection in robotic grasping. By adopting a novel token reorganization strategy, VTF - AVIT substantially improves computational efficiency while enhancing accuracy. The model excels in dynamic and uncertain environments, providing robust and reliable grasping strategies critical for real-time applications in autonomous robotics. Experimental results demonstrate that VTF - AVIT outperforms traditional TranSformer and CNN - LSTM models in both accuracy and computational efficiency. Furthermore, the fusion of tactile and visual data allows the system to effectively adapt to varying environmental conditions and object characteristics, thereby advancing the capabilities of robots in complex manipulation tasks. The code is available at https://github.com/Bugs-Bunny01/VTF-AVIT.