Generalizable Force Estimation Based on Graph Networks for Thin Lensless Vision-Based Tactile Sensor

Vision-based tactile sensors hold significant promise in robot manipulation. However, achieving precise and generalizable force estimation is still challenging due to the nonlinear and heterogeneous nature of the sensor's elastic material. In this work, we propose a novel force estimation network for a thin lensless vision-based tactile sensor we develop with a thickness of less than 1 cm. We employ a graph neural network to model the relationship between surface deformation and contact force. Specifically, we utilize a self-attention mechanism to capture the influence of different nodes on their surrounding nodes. Experimental results on our developed tactile sensor and GelSight Mini demonstrate the effectiveness of our approach, achieving high accuracy even when trained with only one contact shape. Furthermore, our method provides the force distribution on the surface, which is essential for certain robot manipulation tasks.