Evaluating robustness and explainability in Arabic authorship models for detecting AI-generated texts

The rapid development of large language models has intensified the need for reliable methods that distinguish human-authored Arabic text from AI-generated text while remaining interpretable across orthographic, dialectal, and register variation. This study evaluates baseline, transformer, ensemble, and hybrid architectures for Arabic authorship attribution and AI-generated text detection using the AraGenEval corpus and a 10,000-text dialectal Arabic supplement. It combines in-domain evaluation, robustness testing, bootstrap confidence intervals, paired statistical comparisons, feature-based explainability, prediction-level error analysis, and probability-calibration diagnostics. The prediction-level matrix was used as the controlling source for exact confusion counts, class-level metrics, Brier scores, expected calibration error, and error-code distributions. The ensemble of independently fine-tuned AraBERT v2 and XLM-RoBERTa Large produced the strongest in-domain result among the tested systems (accuracy = 94.7%, weighted F1 = .947, macro F1 = .947, MCC = .894, 95% bootstrap CI [.932, .960]), but performance declined on diacritic-stripped text and sharply under dialectal/social-media shift. Exact calibration analysis showed that the Brier score worsened from .046 in-domain to .167 in the dialectal condition, while 10-bin expected calibration error increased to .163 for dialectal data. The hybrid AraBERT-stylometric model numerically exceeded AraBERT in the primary in-domain matrix (weighted F1 = .936 vs. .921), but the paired comparison was not statistically significant (p = .199); consequently, handcrafted stylometric features are interpreted as explanatory aids rather than evidence of reliable predictive superiority. The findings support calibrated, dialect-aware, and uncertainty-reported Arabic AI-text detection.