Decomposing Audio into Timbral Features with Convolutional Neural Networks

AI has gained traction in industry and academia in recent years, and music is no exception. Part of this increase in visibility is due to new methods that process audio recordings directly, rather than relying solely on symbolic representations of music like spectrograms or music notation. These methods commonly use one-dimensional (1D) filter convolution, a well-established signal processing technique built into deep neural networks. In musical AI, 1D convolutional filters analyze digital audio recordings to extract sonic features useful for tasks like music generation or recommendation. Despite 1D filter convolution’s increasing prevalence, though, its impact on AI’s ability to learn musical features is underexplored.This paper addresses these gaps through an epistemic study of AI-based 1D filter convolution, offering two novel contributions. First, experimental results show that 1D filter convolution is heavily biased. It favors features typical of synthetic musical styles, such as electronic dance music, where sounds are highly repetitive and often have simple frequency spectra. However, it struggles to capture the inconsistent and spectrally complex sounds that characterize acoustic styles like folk music, thereby constraining the network’s ability to generalize. Second, this study shows that, despite this bias, 1D filter convolution can effectively parse large audio collections by timbre––broadly defined as the quality of a sound. This finding expands the potential applications of 1D convolutional neural networks and suggests new ways of working with 1D signal data.