Surtify: A Smart Surface Identification System Based on Multi-dimensional Acoustic Dispersion

With the continuous advancement of the smart home market, household items have become more intelligent. By identifying the different material attributes of various household tabletops, we can obtain contextual information and engage in further interactions. This paper explores a phenomenon, acoustic dispersion and applies it to surface identification. Integrating mobile deep learning, we propose a method to recognize solid surface materials through tapping and utilize tapping events as triggers for interaction by identifying surface material categories. To achieve the goal of zero training effort and tiny hardware cost for unseen users, we analyze the propagation patterns of different frequency components in tapping sound signals from both temporal and spatial dimensions. Based on this, we propose a time-frequency data augmentation strategy that combines multiple event points to improve the recognition accuracy. The results show that, using the leave-one-user-out cross validation for 24 volunteers trained with simple guidance on tapping intensity and position, our system achieves a recognition accuracy of 98.6% for 11 common material types. For 11 newly recruited volunteers without specific control over force and position, the accuracy also remains at 93.4%. We also conduct a series of cross-environment experiments and user study, all of which indicate the high potential of our system.