Saudi Arabic automatic speech recognition and dialect identification with calibrated system combination and acoustic–textual modeling

Abstract Saudi Arabic comprises five major regional dialects—Najdi, Hijazi, Khaliji, Janubi, and Shamali—that show substantial lexical, phonological, and prosodic variation, yet remain underrepresented in automatic speech recognition (ASR) and dialect identification (DID). Existing systems are limited by data scarcity, class imbalance, and high similarity among closely related dialects. This study presents a two-phase framework for Saudi Arabic ASR and DID using a curated corpus of 27,940 utterances (~ 34 hours). The first phase fine-tunes four wav2vec 2.0 variants: XLSR-53, XLS-R 300M, XLS-R 1B, and MMS-1B. It further integrates a KenLM language model trained on 2.99 million dialectal lines, dialect-specific orthographic normalization, and calibrated multi-signal reranking by using BERT-based pseudo-log-likelihood scores. The second phase introduces SaudiDID-X, a multimodal DID model that fuses XLS-R 1B acoustic features with SaudiBERT textual features through late fusion. The ASR system achieves 32.16% word error rate (WER), while SaudiDID-X reports 51.63% ± 1.57 macro-F1 for five-class classification and 80.67% for binary classification. Bootstrap testing and three-seed variance estimates support the reliability of the reported results. Confusion-matrix and ablation analyses identify Najdi as a dominant attractor class, absorbing 64.3% of Khaliji and 53.0% of Hijazi misclassifications.