Ultra-Low Memory Speech Denoising using Quantization-Aware Neural Architecture Search

This paper introduces a novel neural network-based speech denoising approach designed for operation on ultra-low memory devices such as, e.g., gaming headsets. We show that neural network models with a memory footprint in the range of 15–30 kB can achieve effective speech denoising performance. First, to this end, a Neural Architecture Search (NAS) approach is proposed in a search space encompassing Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNN s). The goal is to optimize the trade-off between speech denoising performance and memory usage. Experiments indicate that Gated Recurrent Units (GRUs) are particularly prone to quantization issues in tight memory conditions compared to Long Short-Term Memory (LSTM) RNNs, a phenomenon not explored in prior research. Secondly, this work introduces a data preprocessing technique to model the microphone characteristics of the target gaming headset device, which further helps to optimize the speech denoising performance on-device. We believe these findings can serve as future guidance for deploying sophisticated neural network-based speech and audio processing algorithms on resource-constrained edge devices.