Automatic detection of scratching events on vehicles with audio-based spectrograms

The identification of damages, specifically scratches, on the structure of vehicles is a challenge to many businesses. In most automobile industries, this process is mainly performed by human vision, which is unstable and error-prone. Companies are avidly seeking for automatic solutions to facilitate this type of inspection. This article is about an endeavour that intends to develop a novel software-based solution for automatically identifying events that can cause scratches on passenger vehicles. The algorithm that identifies scratching events is at the heart of that solution. The algorithm is based on a convolutional neural network and was designed to use audio data obtained from microphones installed in vehicles. The article presents methods for transforming audio signals into images representing the spectral characteristics of the audio. It also describes the development and the evaluation of the convolutional neural network that was trained to identify scratching events. The convolutional neural network undergoes training, considering the vast array of sounds that a microphone can detect, within the countless everyday driving environments which vehicles can be subjected to. The accuracy of the convolutional neural network that identifies scratching events has an excellent performance, as indicated by a Matthews correlation coefficient equal to +0.90. Our work highlights the potential of convolutional neural networks in scratching events identification and prepares for future research to expand the dataset and to incorporate a wider range of events to be detected. These promising results permit one to consider the use of the algorithm in different applications for the automotive domain. • Scratch detection within the context of vehicular audio is possible. • CNNs distinguish the acoustic signatures of scratches by analysing images. • An algorithm for scratch detection in vehicles handles realistic scenarios. • Deep Learning approaches are used for sound classification and detections tasks. • The CNN-based algorithm achieved MCC scores of over 0.90 for the best models.