YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection

Dysfluent speech detection is the bottleneck for disordered speech analysis and spoken language learning. Current state-of-the-art models are governed by rule-based systems [1, 2] which lack efficiency and robustness, and are sensitive to template design. In this paper, we propose <i>YOLO-Stutter</i>: a <i>first end-to-end</i> method that detects dysfluencies in a time-accurate manner. YOLO-Stutter takes <i>imperfect speech-text alignment</i> as input, followed by a spatial feature aggregator, and a temporal dependency extractor to perform region-wise boundary and class predictions. We also introduce two dysfluency corpus, <i>VCTK-Stutter</i> and <i>VCTK-TTS</i>, that simulate natural spoken dysfluencies including repetition, block, missing, replacement, and prolongation. Our end-to-end method achieves <i>state-of-the-art performance</i> with a <i>minimum number of trainable parameters</i> for on both simulated data and real aphasia speech. Code and datasets are open-sourced at https://github.com/rorizzz/YOLO-Stutter.