Exploring Deep Object Content through In-hand Tactile Vibration Sensing with Shaking Manipulation

Tactile sensing provides robots with insights beyond visual limitations, enabling the detection of surface characteristics and deformation properties like texture and slip. However, identifying internal content conditions remains a challenge. This study introduces a tactile-driven approach for classifying the internal content of various containers through vibration sensing and controlled shaking manipulation. By dynamically altering the mass distribution within the object, the system generates distinct changes in the center of mass (CoM), resulting in unique vibrational patterns. These patterns are captured and processed by a Convolutional Neural Network (CNN) to categorize the content state as depleted, partially loaded, or maximally loaded. Experimental results demonstrate a classification accuracy of 93 %, with clear clustering of content conditions, validating the effectiveness of this tactile-based approach across different fill levels. This tactile-based method significantly enhances robotic perception in tasks requiring internal content knowledge, such as sorting, recycling, and storage, offering a robust and scalable solution for real-world applications.