MTU: A Melody Extraction Method Using Swin U-Net Enhanced with Mamba Blocks

Mamba-Transformer U-Net (MTU) is a melody extraction method designed for polyphonic audio, addressing both vocal and non-vocal music tracks. Its architecture integrates Swin U-Net with Mamba blocks, enhancing feature extraction and achieving multi-scale representation of intricate polyphonic music signals. A key innovation of MTU is the adoption of multiple patch sequences with varied orders within Mamba blocks, significantly improving melody extraction accuracy. Furthermore, a preprocessing technique involving a Gaussian filter smooths the target salience map, refining melody labeling on the spectrum and boosting training effectiveness. Comprehensive evaluations on benchmark datasets, including ADC2004, MIREX05, and MedleyDB, indicate that MTU performs strongly across key metrics such as pitch and chroma accuracy, highlighting its potential as an effective solution for melody extraction in challenging polyphonic scenarios.