Robust Automatic Stuttering Detection Through Multi-Domain Feature Learning and Attention-Based Classification

Automatic detection of speech disfluencies, particularly stuttering, remains a challenge because of the variability of disfluency patterns, limited annotated pathological speech data, class imbalance, and diverse acoustic conditions. This study proposes a robust deep learning framework for automatic detection and classification of speech disfluencies, including repetitions, prolongations, blocks, interjections, and fluent speech. The proposed framework combines the balanced data augmentation with multi-domain feature extraction to capture the complementary spectral, temporal, pitch, and contextual characteristics of speech. Mel-frequency cepstral coefficients (MFCCs), zero-crossing rate (ZCR), spectral flux, and contextual representations are derived from the pre-trained Wav2Vec 2.0 are integrated, to improve the representation of disfluent speech. To improve the discriminative feature learning, the extracted features are processed using an attention-based convolutional architecture, while employing the class-balancing strategies to reduce the influence of the dominant fluent-speech class. Experiments were conducted using the benchmark datasets like UCLASS, SEP-28k, and FluencyBank to demonstrate the effectiveness of the proposed approach across different speech conditions. Feature-level evaluation shows an improvement in accuracy from 82.3% using MFCCs alone to 91.2% using the complete feature set, with precision, recall, and F1-score of 0.90, 0.91, and 0.91, respectively. For disfluency-specific detection, the proposed model achieved accuracies of 95.60%, 94.50%, and 60.80% for repetitions, prolongations, and blocks on UCLASS, and 86.40%, 92.50%, and 74.20% on SEP-28k, respectively. An ablation study further demonstrates the contribution of data augmentation, contextual feature extraction, feature selection, attention, and class-balancing components, with the complete framework achieving 95.10 ± 0.38% accuracy and a 94.35% macro F1-score. These results demonstrate the potential of the proposed framework for accurate and robust automatic speech disfluency detection across diverse speech datasets.