Speaking with Robots in Noisy Environments

A fundamental limitation for speech-enabled human-robot interaction (HRI) is automatic speech recognition (ASR), or the process of converting a raw speech signal into text. If this fails then all downstream tasks will fail. While commercial ASR systems have improved substantially in recent years, they are still well known to struggle with noisy speech, which is commonplace in a variety of social, medical, and military environments. In this paper, we address this challenge by introducing a dataset for training and evaluating ASR systems to be robust to background noise. The dataset is comprised of 100 hours from the Librispeech ASR Corpus, which we supplemented with a variety of noises, including office, traffic, crowd, and others. We evaluate the state-of-the-art Whisper ASR model on the dataset and demonstrate improved ASR accuracy on noisy speech. We also investigate how several methods of speech enhancement can be trained on the data to further improve performance. Finally, we demonstrate this approach in a real-world HRI task involving noisy speech. The dataset and models are available for researchers to freely use to improve the performance of their ASR systems for HRI in noisy environments.