Validity and Robustness of Denoisers: A Proof of Concept in Speech Denoising

We can use randomized smoothing to assess and certify the robustness of Machine Learning models. It uses a norm ball around inputs to check if the model's output remains consistent. Adding a smoothing process, like a signal denoiser, can enhance the model's robustness against adversarial attacks. However, the robustness of the smoothing process itself is an open question, especially as denoisers are getting more complex. This study proposes a method to validate and certify the robustness of signal denoisers. Using a baseline and a user-defined noise model, we synthesize a significant noise pattern unseen by the denoiser. This pattern forms two balls around the baseline and the denoised signal, whose volume ratio quantifies the model's robustness. We applied this framework to discriminative and generative models for speech denoising, demonstrating its practicality. Our evaluations of the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Demucs</small> architecture, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RNNoise</small>, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Resemble+Enhance</small>, and <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VoiceFixer</small> show that we can identify robust denoisers under different conditions and gain insights into the workings of black-box denoisers. This methodology offers a comprehensive metric for model comparison.