Comparative Evaluation of Explanation Reliability in Shipment-Level Supply Chain Risk Classification: A Leakage-Checked Benchmark of XGBoost and a Hybrid XGBoost–Neural Framework with SHAP and LIME
Machine learning can forecast supply chain disruptions, but black-box models limit transparency, and the reliability of post-hoc explanations is rarely evaluated. We provide a leakage-checked, reproducible comparative benchmark for shipment-level risk classification on a public synthetic dataset of 5000 shipments, comparing XGBoost alone with a hybrid XGBoost–neural model (XAI-SC-RiskNet), both explained with SHAP and LIME. The hybrid (71.60% accuracy, ROC-AUC 0.819) is statistically indistinguishable from XGBoost (72.60%, 0.819; McNemar’s test p=0.123), as repeated cross-validation and a chronological hold-out confirm. On 500 test shipments, the Local Explanation Consensus Index (LECI; top-6 overlap of local SHAP and LIME features) and LIME stability are lower for the hybrid (0.57 versus 0.68; 0.80 versus 0.88), so the hypothesis that hybridization stabilizes explanations is not supported. LECI was modestly associated with misclassification (AUC 0.61) but added no significant information to predictive uncertainty (AUC 0.77); it is therefore a descriptive measure of explanation agreement, which does not establish explanation correctness or better decisions, and not a validated quality-control mechanism. These findings are specific to one synthetic benchmark with a concentrated weather signal and do not show that hybrid architectures are generally inferior.
Authors
- Ayan Barua (ORCID: https://orcid.org/0009-0009-8060-6339)
- Khandakar Rabbi Ahmed (ORCID: https://orcid.org/0000-0003-0235-2870)
- Shamimur Rahman (ORCID: https://orcid.org/0009-0007-1676-168X)
- Sakib Salam Jamee (ORCID: https://orcid.org/0009-0006-7694-8641)
- Muslima Begom Riipa (ORCID: https://orcid.org/0009-0008-4473-669X)
- Mainul Islam Khan (ORCID: https://orcid.org/0009-0006-2383-1618)
- Md Shahidul Islam (ORCID: https://orcid.org/0009-0008-7483-9294)
Institutions
- Trine University (US)
- International University of Business Agriculture and Technology (BD)
- International American University (US)
- Stanford University (US)
Publication Details
- Journal
- Information
- Published
- 2026-11-19
- DOI
- https://doi.org/10.3390/info17111181
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- article
- Field-Weighted Citation Impact
- 0.00