Critical Assessment of the Disparity Between Quantitative and Qualitative Performance of U-Net Based Audio Source Separation

The separation of audio sources with the application of deep learning techniques has become a hottopic in research due to growing interest in this area. UNet [1] architectures, in particular, have beenable to provide strong quantitative results. Nevertheless, even after achieving good scores onevaluation metrics such as SDR or SIR, the relative quality of the separated audio is not good. Inthis paper, we look into the quantitative versus qualitative performance difference in UNet-basedaudio source separation. We consider the effects of spectral smoothing, phase reconstructionproblem, and the more general issue of traditional loss functions that convert everything intonumbers and ignore everything else. The research explains that the structure of UNet providesblurring of the high-frequency details and adds artifacts that reduce the naturalness of the separatedsound. In addition, we devise other ways to enhance qualitative performance like incorporatingperceptual loss functions, building hybrid systems, and robust post-processing. For deep learningaudio separation, the models developed by us highlight the need for incorporating psychoacousticapproaches in bordering objective metrics with subjective hearing perception.