Globally Normalizing the Transducer for Streaming Speech Recognition

The Transducer (e.g. RNN-Transducer or Conformer-Transducer) generates an output label sequence as it traverses the input sequence. It is straightforward to use in streaming mode, where it generates partial hypotheses before the complete input has been seen. This makes it popular in speech recognition. However, in streaming mode the Transducer has a mathematical flaw which, simply put, restricts the model’s ability to change its mind. The fix is to replace local normalization (e.g. a softmax) with global normalization, but then the loss function becomes impossible to evaluate exactly. A recent paper proposes to solve this by approximating the model, degrading performance. Instead, this paper proposes to approximate the loss function, allowing global normalization to apply to a state-of-the-art streaming model. Global normalization reduces its word error rate by 9–11% relative, closing almost half the gap between streaming and offline mode.