An N-Best List Selection Framework for ASR N-Best Rescoring

Inspired by advancements in deep learning, automatic speech recognition (ASR) systems have experienced significant growth. To further enhance performance, many ASR systems focus on refining model architecture or incorporating additional linguistic and semantic information. For the latter, many studies have explored various post-processing methods to leverage the information. Currently, ASR post-processing primarily revolves around two approaches: n-best rescoring and error correction. Orthogonal to these attempts, we propose a novel idea of selecting a new hypothesis list to enhance the accuracy of existing n-best rescoring models. Our modeling philosophy is based on the assumption that a well-constructed hypothesis list should be diverse. Therefore, the new set of n-best lists is expected to include better hypotheses, allowing the current n-best rescoring models to more effectively distinguish the optimal hypothesis. We evaluate the efficiency and effectiveness of the proposed methods by implementing the causal language model, the masked language model, and the BERT-based rescoring model on the AISHELL-1 dataset.