Multidirectional slip detection and avoidance using dynamic 3D tactile meshes from visuotactile sensors

Visuotactile sensors have gained attention during the last years in robotics because they are able to reconstruct with high precision the 3D contact shape (or mesh) between the robotic fingers and the object. A new slip detection and avoidance algorithm is proposed based on the dynamic variation of the height of the contact mesh. Firstly, the contact mesh is reconstructed in real time by applying a neural network that estimates normal vectors from color variations along all the pixels of the images recorded by the camera inside the tactile sensor. The contact mesh corresponding to this height map is used for detecting slip with higher success rates in comparison with previous approaches based on machine learning methods directly applied to contact images or the analysis of markers integrated into the sensor’s surface. The proposed algorithm is validated experimentally in multiple directions not only for different types of objects (volumetric/planar/linear, deformable/rigid) but also with different resolutions of the contact mesh.