Aggregation-Free Uncertainty Estimation for CTC-Based Automatic Speech Recognition

Nowadays, deep neural networks (DNNs) are utilized in many assistance systems of various safety-critical relevance and have even established themselves in high-stakes decision making, such as medical purposes. Therefore, an accurate uncertainty estimation of the networks output is crucial for the system's functional reliability in operational contexts. In this work, we investigate predictive uncertainty estimation for connectionist temporal classification (CTC) based automatic speech recognition models. We compute an uncertainty estimate based on the intermediate variables of the CTC Forward- Backward algorithm for any sub-sequence length, including token- and word-level, without the necessity to aggregate frame level uncertainties. Our approach outperforms prior state-of-the-art methods based on frame-level probability or entropy, as well as ensemble-based methods for prediction of recognition errors. We evaluate on TIMIT and Mozilla Common Voice (MCV) using pre-trained models of different model architectures. Our approach identifies 80% of a wav2vec2.0 model's errors on MCV by selecting less than 9% of the tokens.