Deceptive Overgeneralization: When Adaptive Learning Enables Systematic Misapplication

ABSTRACT Background Adaptive learning systems infer mastery from observed correctness, but this approach may fall short when learners exhibit ‘deceptive overgeneralization’—performing correct actions based on incomplete understanding. This phenomenon can mislead systems into prematurely stopping practice, leaving knowledge gaps unaddressed. Clarifying when deceptive overgeneralization occurs and how to address it can improve the precision of adaptive learning technologies. Objectives This research empirically examines whether deceptive overgeneralization occurs in adaptive learning, assesses its consequences for mastery‐based stopping rules, and evaluates a systematic procedure for detecting and remedying it. Methods We conducted 11 experiments using Intelligent Tutoring Systems (ITSs) for Riichi Mahjong across multiple semesters with university students (total N = 192). The studies employed a pretest–posttest design to assess ITS effectiveness, and within‐subjects designs with adaptive and full‐exposure conditions to examine overgeneralization and remediation. Tailored remediation consisted of targeted ‘do‐not‐act’ problems with feedback that names the missing application constraints. Results and Conclusions ITSs improved learning outcomes, but learners frequently misapplied acquired actions in contexts where they should have refrained, indicating deceptive overgeneralization. Under adaptive learning, mastery stopping rules often ended practice before learners encountered such ‘do‐not‐act’ contexts, leaving the issue undetected. Tailored remediation using targeted ‘do‐not‐act’ problems with constraint‐naming feedback reduced misapplication to near‐floor levels across skills and cohorts, highlighting a limitation of correctness‐based mastery inference and a practical method to strengthen mastery assessment for knowledge compilation. A secondary analysis of K–12 decimal learning data further illustrates how the framework applies in a traditional educational domain.