Fast Sampling Based on Policy Gradient for Diffusion-Based Speech Enhancement

The effectiveness of diffusion-based generative models in speech enhancement tasks has been reported. These generative methods can produce high-quality audio with less distortions and outperform their discriminative counterparts, especially in mismatched conditions. However, due to the multi-step denoising property, these models usually have slower audio generation speed and a tendency to accumulate errors in the enhancement process. To address these issues, we propose a novel approach that considers the reverse process as a sequential decision-making task and constructs the distribution similarity metric into the training objective. In this way, the generator can learn data distributions well and explore more efficient sampling trajectories. The results of our experiments indicate that our proposed method not only reduces the number of steps required for audio generation, but it also produces even higher quality samples and preserves the generalizability to unknown noise.