Enhancing DeepIRT through counterfactual item-invariance regularization and joint learning

Knowledge Tracing (KT) models are widely used in adaptive learning systems to estimate students’ evolving knowledge states. Despite recent advances in deep KT models, many approaches exhibit poor probabilistic calibration, producing overconfident predictions that misrepresent students’ knowledge. This paper presents Counterfactual Item-Invariance Regularized Knowledge Tracing (CIR-KT), a bias-aware approach for calibrated knowledge tracing. For each interaction the method constructs an item-level counterfactual, substituting the presented item while holding the observed response fixed, and enforces consistency between the factual and counterfactual latent states through a regularization objective. By reducing the model’s reliance on item-identity shortcuts, the framework improves probability calibration without altering the underlying KT architecture. On a severely miscalibrated private College of Engineering (COE) subset, counterfactual regularization substantially reduces Expected Calibration Error (ECE) and Brier score and improves ranking (AUC and PR-AUC) relative to the DeepIRT backbone, while on already well-calibrated public benchmarks its effect on discrimination and calibration is small. Skill-level analyses show that the calibration gains are broad-based across knowledge components rather than driven by a few items. These results indicate that item-level counterfactual regularization is a practical mechanism for improving calibration in knowledge tracing, supporting more dependable adaptive learning systems. At the validation-selected operating point, CIR-KT is the best-calibrated model among the knowledge-tracing backbones evaluated while remaining competitive in AUC, and it provides interpretable ability and difficulty estimates that attention baselines do not.

Authors

Institutions

Publication Details

Journal
Frontiers in Artificial Intelligence
Published
2026-11-12
DOI
https://doi.org/10.3389/frai.2026.1939804
Primary Topic
Intelligent Tutoring Systems and Adaptive Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Enhancing DeepIRT through counterfactual item-invariance regularization and joint learning

Xishuang Dong, Pamela Obiomon, Lijun Qian, Christian Onyeke
Frontiers in Artificial Intelligence
Intelligent Tutoring Systems and Adaptive Learning
article

Enhancing DeepIRT through counterfactual item-invariance regularization and joint learning

Xishuang Dong, Pamela Obiomon, Lijun Qian, Christian Onyeke
article en

Abstract

Knowledge Tracing (KT) models are widely used in adaptive learning systems to estimate students’ evolving knowledge states. Despite recent advances in deep KT models, many approaches exhibit poor probabilistic calibration, producing overconfident predictions that misrepresent students’ knowledge. This paper presents Counterfactual Item-Invariance Regularized Knowledge Tracing (CIR-KT), a bias-aware approach for calibrated knowledge tracing. For each interaction the method constructs an item-level counterfactual, substituting the presented item while holding the observed response fixed, and enforces consistency between the factual and counterfactual latent states through a regularization objective. By reducing the model’s reliance on item-identity shortcuts, the framework improves probability calibration without altering the underlying KT architecture. On a severely miscalibrated private College of Engineering (COE) subset, counterfactual regularization substantially reduces Expected Calibration Error (ECE) and Brier score and improves ranking (AUC and PR-AUC) relative to the DeepIRT backbone, while on already well-calibrated public benchmarks its effect on discrimination and calibration is small. Skill-level analyses show that the calibration gains are broad-based across knowledge components rather than driven by a few items. These results indicate that item-level counterfactual regularization is a practical mechanism for improving calibration in knowledge tracing, supporting more dependable adaptive learning systems. At the validation-selected operating point, CIR-KT is the best-calibrated model among the knowledge-tracing backbones evaluated while remaining competitive in AUC, and it provides interpretable ability and difficulty estimates that attention baselines do not.

Frontiers in Artificial IntelligenceVol. 9
Prairie View A&M University (US)
Quality education
Intelligent Tutoring Systems and Adaptive Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon