Reliability of Urdu–Arabic Abusive Language Detection with Cross- Lingual Transformers and Explainable AI

Abstract Automated detection of abusive language remains challenging for Urdu and Arabic, and high classification scores do not establish that a model transfers reliably or explains its decisions faithfully. We evaluate a bilingual XLM-RoBERTa detector trained on 65,344 records, combining Integrated Gradients (IG), KernelSHAP, and an open-weight Aya-23-8B explainer with tests of cross-lingual transfer, robustness, calibration, and context sensitivity. The classifier achieved 0.821 accuracy (Urdu, 0.816; Arabic, 0.826), with mean accuracy of 0.819 ± 0.001 across three seeds. Zero-shot transfer between Urdu and Arabic remained near chance (0.50–0.53), indicating that the small in-language performance gap depended on joint bilingual training. IG and KernelSHAP showed moderate agreement (pooled Spearman ρ = 0.369). Although Aya-23 explanations echoed 69% of attributed words, deletion tests found that quoted words were not more decisive than random words, whereas IG-selected words were. Synthetic Roman-Urdu transliteration reduced Urdu accuracy by 0.218. A contrast set sharing sensitive keywords exposed context-insensitive predictions (accuracy, 0.571); context-aware fine-tuning raised contrast accuracy to 0.857 while leaving full-test accuracy nearly unchanged. These findings support evaluating multilingual abusive-language detectors through targeted reliability tests rather than aggregate accuracy alone.