Enhancing End-to-End Speech Recognition Models with a Joint Decoding Strategy

Automatic Speech Recognition (ASR) is a crucial technology within Artificial Intelligence, with end-to-end ASR gaining significant attention for its outstanding performance. However, end-to-end ASR models still have deficiencies in capturing local and global information of text, and especially perform poorly in long-form speech recognition. To address these issues, this paper proposes a new joint decoding strategy that combines the advantages of CTC decoder and intermediate CTC loss to enhance the local feature representation of the model and accelerate the training process. Meanwhile, we design a new AED decoder, named MA decoder, which is able to better capture global and local information by combining the advantages of RNN-T and Transformer decoders and introducing MA (Mamba Augmented) module. The combined experimental results demonstrate that the proposed joint decoding strategy performs effectively on the Aishell-1 and LibriSpeech datasets, with significant performance improvements in long-form speech recognition tasks on the TEDLIUM2 dataset.