A Concept-Bottleneck Explainable AI Framework for Diagnosing Agile Delivery Outcomes

Agile outcome models commonly map Jira variables directly to a retrospective label and then explain the prediction through fragmented feature attributions; they rarely separate domain concepts, team clustering, unresolved work, and concept-label coupling. This study evaluates a domain-informed, concept-bottleneck-style explainable AI architecture for retrospective diagnosis of Agile Epic outcomes. A frozen Jira export of 10,000 unique issue-level records was linked to a pre-specified analytical cohort of 180 Epics across 14 teams. Six experts rated efficiency, effectiveness, sustainability, and contextual risk, while outcomes were recorded as Successful, Challenged, or Unsuccessful. Because the outcome labels and concept ratings were informed by the same Jira evidence, the models estimate consistency with an expert labelling procedure, rather than independent project success. Under five-fold group-aware cross-validation, the fixed-configuration flat LightGBM achieved macro-F1 = 0.864 ± 0.053 and the fixed-configuration HMXAI/CBM-style model achieved 0.843 ± 0.084. These descriptive primary scores are not a joint nested-model-selection comparison. The proposed method, therefore does, not demonstrate a performance improvement; its contribution is an inspectable diagnostic structure. Performance fell materially on the resolved-only subset (LightGBM macro-F1 = 0.645), and model-specific nested, leave-one-team-out, calibration, uncertainty, correlation, and intervention analyses further bound the claims. Concept interventions were not uniformly monotone, so the concept layer is domain-interpretable in form but not yet user-validated as actionable. The study contributes a transparent audit of when concept-level diagnosis can complement flat classification and when circularity, censoring, and shortcut learning restrict interpretation.