Speckle Skin‐Based Multimodal Tactile Perception for Fine Robotic Manipulation

Tactile sensing is crucial for providing physical interaction between the robotic hand and objects, enabling effective and safe grasping and manipulation. Vision‐based tactile sensors have gained increasing popularity due to their high spatial resolution and extensibility. Currently, most vision‐based tactile sensors utilize either the marker array tracking method or the photometric stereo method. Spatial resolution of the marker array is constrained by the density of markers, while the photometric stereo approach requires complex illumination. Herein, we propose a miniature vision‐based tactile sensor exploiting a speckle skin, namely, SpeckleTac . By applying a dense optical flow method with adaptive Kalman filtering and dynamic weight fusion, high‐resolution and accurate 2D displacement fields are obtained, and subsequently the reconstruction of the 3D surface. Then, contact object classification and pose estimation are realized through a ResNet‐based architecture, SPTacNet, integrated with squeeze‐and‐excitation (SE) residual blocks. Furthermore, by incorporating a TV‐regularized iFEM method, contact pressure distribution and 6‐axis force and torque are obtained. We demonstrate that the SpeckleTac can accurately locate the centroid of an unknown object, enabling stable grasping and facilitating safe handling of blood samples in glass tubes. This work provides a promising solution for multimodal tactile perception of robotic hands in “real‐world” applications.