A Real‐Time Imaging Sensing System to Visualize Elastomer Surface Profile Evolution for Dynamic Tactile Recognition

Abstract Elastomer‐based interfaces provide rich functionalities for tactile sensing, particularly in making tiny differences in contact dynamics potentially detectable. However, the minimal motion‐induced changes in the elastomer's surface and their disappearance during time‐lapse limit the state recognition within the current scheme of motion recognition. In this work, a new scheme of real‐time motion mode recognition for subtle deformations is proposed, which uses an optical tactile sensing system to visualize and distinguish tiny variations in surface profile evolution encoded as images. Illustrating with a sphere, sliding‐induced asymmetric elastomer surface deformation is visualized as a “drag” in optical images. The convolutional neural network (CNN) algorithm is used to analyze the evolution of surface contour features during the interaction between the sphere and the elastic medium. Motion state recognition is achieved with 80% accuracy when a displacement of only 8.3% of the sphere diameter is produced. In addition, the system also offers the potential to analyze dynamic motion information through a single image, with an accuracy of 82.7% for velocity recognition. This dynamic real‐time recognition framework for soft media deformation paves the way for novel motion‐based input commands for tactile sensing and human‐computer interaction applications.