TF-DiffuSE: Time-Frequency Prior-Conditioned Diffusion Model for Speech Enhancement

The exceptional generative capabilities of conditional diffusion probabilistic models have demonstrated significant advantages in speech enhancement (SE). However, the introduction of Gaussian noise during the data-to-noise diffusion process fails to consider the critical time-frequency (T-F) structure of audio signals, hindering the quality of generated speech. Moreover, diffusion models typically require numerous denoising steps to achieve high sample quality, leading to increased computational time compared to other generative methods. To address these issues, this paper proposes an efficient T-F prior-conditioned diffusion model for SE, referred to as TF-DiffuSE, which incorporates a fast sampling technique. Specifically, in TF-DiffuSE, a guidance network trained to capture the speech energy distribution is introduced to obtain the T-F prior guidance for the diffusion process. Moreover, a hybrid sampler method is introduced, significantly reducing the number of sampling steps while maintaining speech quality. Comprehensive experimental results demonstrate that the proposed model outperforms existing diffusion-based SE models, achieving superior enhancement performance and exhibiting good generalization in unseen scenarios.