EXPLAINABLE ARTIFICIAL INTELLIGENCE FOR FORMATIVE SUPPORT ALLOCATION IN HIGHER EDUCATION: A HUMAN-SUPERVISED DECISION ARCHITECTURE AND SYNTHETIC SIMULATION

Artificial intelligence and learning analytics can identify patterns of learning risk earlier and support more differentiated allocation of formative support in higher education. Yet predictive accuracy does not by itself make an educational decision valid: evidence must be interpretable in relation to learning goals, explanations must not be confused with causal diagnosis, and final support decisions should remain subject to professional review and learner participation. This article develops a human-supervised decision architecture and computationally illustrates its predictive and support-allocation components through a reproducible simulation of 10,000 synthetic learner-course observations without using personal data. Logistic regression, random forest, and XGBoost were compared under class imbalance. Decision thresholds were selected on a validation subset by minimizing a weighted decision-cost function in which missing an at-risk learner was assigned greater weight than generating an additional support-review action. On the independent test set, XGBoost achieved ROC-AUC = 0.910, PR-AUC = 0.764, recall = 0.902, F1 = 0.612, and Matthews correlation coefficient = 0.526 at a threshold of 0.220. Mean absolute SHAP values indicated that formative assessment score, task completion, attendance, prior achievement, self-regulation, engagement, learning progress, and study consistency contributed most strongly to predictions. Under the stated scenario assumptions, the AI-assisted allocation strategy produced an illustrative weighted decision cost that was 62.4% lower than a reactive end-of-term strategy and 26.9% fewer review/support actions than a simple rule-based strategy. The scenario models alert allocation and review workload only; it does not model teacher acceptance, intervention implementation, learner response, or learning effects. The methodological contribution is the integration of cost-sensitive thresholding, a mandatory teacher decision gate, SHAP-informed pedagogical reason codes, learner challenge and data correction, and workload-aware threshold governance within one auditable formative-assessment architecture. The reported metrics therefore illustrate the predictive and allocation logic of the architecture rather than the effectiveness of the pedagogical system as a whole.

Authors

Institutions

Publication Details

Journal
Education Innovation Practice
Published
2026-10-30
DOI
https://doi.org/10.31110/2616-650x-vol14i8-017
Primary Topic
Online Learning and Analytics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

EXPLAINABLE ARTIFICIAL INTELLIGENCE FOR FORMATIVE SUPPORT ALLOCATION IN HIGHER EDUCATION: A HUMAN-SUPERVISED DECISION ARCHITECTURE AND SYNTHETIC SIMULATION

Gunel Mammadova
2 links
Education Innovation Practice
Online Learning and Analytics
article

EXPLAINABLE ARTIFICIAL INTELLIGENCE FOR FORMATIVE SUPPORT ALLOCATION IN HIGHER EDUCATION: A HUMAN-SUPERVISED DECISION ARCHITECTURE AND SYNTHETIC SIMULATION

Gunel Mammadova
article en

Abstract

Artificial intelligence and learning analytics can identify patterns of learning risk earlier and support more differentiated allocation of formative support in higher education. Yet predictive accuracy does not by itself make an educational decision valid: evidence must be interpretable in relation to learning goals, explanations must not be confused with causal diagnosis, and final support decisions should remain subject to professional review and learner participation. This article develops a human-supervised decision architecture and computationally illustrates its predictive and support-allocation components through a reproducible simulation of 10,000 synthetic learner-course observations without using personal data. Logistic regression, random forest, and XGBoost were compared under class imbalance. Decision thresholds were selected on a validation subset by minimizing a weighted decision-cost function in which missing an at-risk learner was assigned greater weight than generating an additional support-review action. On the independent test set, XGBoost achieved ROC-AUC = 0.910, PR-AUC = 0.764, recall = 0.902, F1 = 0.612, and Matthews correlation coefficient = 0.526 at a threshold of 0.220. Mean absolute SHAP values indicated that formative assessment score, task completion, attendance, prior achievement, self-regulation, engagement, learning progress, and study consistency contributed most strongly to predictions. Under the stated scenario assumptions, the AI-assisted allocation strategy produced an illustrative weighted decision cost that was 62.4% lower than a reactive end-of-term strategy and 26.9% fewer review/support actions than a simple rule-based strategy. The scenario models alert allocation and review workload only; it does not model teacher acceptance, intervention implementation, learner response, or learning effects. The methodological contribution is the integration of cost-sensitive thresholding, a mandatory teacher decision gate, SHAP-informed pedagogical reason codes, learner challenge and data correction, and workload-aware threshold governance within one auditable formative-assessment architecture. The reported metrics therefore illustrate the predictive and allocation logic of the architecture rather than the effectiveness of the pedagogical system as a whole.

Education Innovation PracticeVol. 14(8)
Azerbaijan State Pedagogical University (AZ)
Quality education
Openalex Percentile: Top 7%
Online Learning and Analytics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon