Coverage-Guided Testing for Automatic Speech Recognition Systems

Automatic Speech Recognition (ASR) systems have become essential in modern life, powering human-computer interactions and voice-activated virtual assistants. These systems, which predominantly rely on deep neural networks, exhibit a probabilistic nature that can lead to unpredictable and erroneous outputs, particularly in varied and dynamic real-world settings. Consequently, it is crucial to thoroughly test ASR systems to identify and resolve bugs that may impair functionality and user experience. Current works for ASR testing fall short in diversity and coverage. To address this, we propose a novel coverage-guided test method for ASR utilizing four neuron coverage criteria to guide the generation of diverse test data and ensure comprehensive testing across multiple dimensions. We evaluated the effectiveness of our method using three popular datasets and two DNN-based ASR systems. The results show that the generated data significantly improve coverage rates, enabling more comprehensive testing.