Uncertainty-Aware Reinforcement Learning for Efficient Adaptive Educational Assessment

Adaptive assessment is efficient only if the system knows how uncertain it is about the learner, because that uncertainty decides which question is worth asking and when to stop. Reinforcement-learning (RL) assessment agents often derive it from Monte Carlo (MC) dropout over a neural knowledge encoder and reward its reduction. We study uncertainty-aware RL for efficient assessment by combining a calibration-aware Bayesian knowledge-state filter (assumed-density filtering on a probit item-response model with knowledge-component deviations and item-calibration noise), a target-aware information criterion with a closed-form expected covariance contraction, and a PPO-trained termination policy with an uncertainty-based state, a per-question cost, a policy-invariant shaping reward and a cross-fitted training environment. On ASSISTments 2009 and Junyi Academy, the selector significantly outperforms random selection and, from ten questions on, a re-implemented HARLA-ED operational agent under both a Bayesian and the agent's own neural estimator (NLL 0.552 vs. 0.558 after 20 questions on ASSISTments) and is statistically tied with mutual-information selection. In our re-implementation, the agent’s MC-dropout uncertainty grows as responses accumulate (0.10 to 0.18), its uncertainty-reduction reward did not improve validation likelihood in any run, and stopping on it needs 1.7–2.5 times as many questions as a fixed-length test of equal accuracy. The learned termination policy matches fixed-length efficiency (5.1–5.4 vs. 5 questions) without threshold tuning, and a value-of-information analysis explains why larger learner-specific savings are hard to obtain in replay protocols.