Endpoint Detection for Streaming ASR with Phoneme Modeling

Endpoint detection in speech recognition identifies when a natural sentence stops during recognition. Recent research on integrating endpoint detection and speech recognition into a single streaming ASR model is limited, with past methods often having high detection latency due to coarse modeling granularity. Therefore, we propose using phoneme information to model endpoint detection to assist the speech recognition system, achieving lower latency in endpoint detection for RNN-T-based streaming ASR systems. Validation on end-to-end endpoint detection-oriented speech recognition tasks shows that, compared to baseline methods, the phoneme-branch-based ASR system significantly reduces endpoint detection latency, improves coverage, and maintains speech recognition performance at the cost of only a slight increase in parameters and real-time factor.