Retrieval-based automatic mashup generation with deep learning-guided features

Pop song mashups have emerged as a prominent research direction in the field of creative music information retrieval (MIR). The goal of a successful mashup system is to create a mashup, a creative synthesis of two or more pop songs that maintains perceptual fidelity to the original tracks. We propose a retrieval-based mashup generation pipeline that identifies compatible source materials based on deep learning-guided features from an extensible collection of contemporary pop songs. The selection heuristic is rooted in the mashability score developed by Davies et al. (2014). It is further enhanced with recent advancements in music information retrieval, self-supervised latent features with autoencoders, and precomputation of normalized spectral features to ensure efficient large-scale searches. Mashups are crafted by swapping stems between the chosen input and compatible tracks, followed by mastering with equalization (EQ) adjustments predicted from the original tracks. The effectiveness of our system is evaluated through a subjective listening test, which measures the perceptual similarity between the original tracks and the mashups, the accuracy of the compatibility heuristic in predicting user preferences, and the system's performance in comparison to established mashup services like RaveDJ.