Self-Information Guided Speech Segmentation for Efficient Streaming ASR

Unlike modern streaming Automatic Speech Recognition (ASR) systems which segment speech into fixed-length chunks for decoding, humans perceive speech in variable-length units of information. This paper proposes a novel method that leverages self-information, a measure of the information contained within an utterance, as a supervisory signal for speech segmentation. By using self-information to guide segmentation, our method creates more informative speech segments than the simpler approach based on word count, achieving faster and more accurate decoding. Our approach is evaluated on English and Japanese datasets, and notably outperforms a topline RNN-Transducer at smaller chunk sizes when predicting fluent transcripts for spontaneous Japanese speech, highlighting its advantage in real-time streaming ASR. This work marks the first application of linguistic self-information to speech segmentation, and contributes towards a more human-like and efficient approach for streaming ASR decoding.