Interpretable Deep Learning for Music Genre Classification Using Spectrogram Analysis

Recognizing and categorizing musical genres is a well-known and long-standing problem in music information retrieval, generally regarded as a black box problem among deep learning pipelines. This paper presents a new interpretable architecture for genre classification by using CNNs, and combining with the novel scripts related to explainability, to extract meaning from spectrograms in order to differentiate genres of music. We used the GTZAN dataset, and implemented three architectures–VGG19, ResNet18 and a custom CNN– and fine-tuned each model using the Mel spectrograms as input. Importantly, we inserted explainability via Grad-CAM, which provided explanatory visualizations, showing transparency into what each model is doing internally to arrive at their final decisions. The most successful performing model was VGG19, which yielded a peak validation accuracy of around 82%, followed by a Custom CNN model at 77.50%, while the ResNet18 had a moderate performance of 75.50%. ResNet18 model exhibited unique interpretability as seen in Grad-CAM. The Grad-CAM visualizations revealed interesting learning behavior: the VGG19 and Custom CNN models were often activated on the negative space in high-frequency regions, which was particularly for blues tracks, which suggests some sort of structural attention beyond the rhythmic contours. The ResNet18 model relied more on some consistent texture patterns. This study provides a general understanding of which frequency regions are most implicated in genre categorization which serves as an important link between the success of deep learning and human understanding of music.