Judgments of learning in the wild: Establishing ecological validity with an intelligent tutoring system in a field study

The monitoring of one's own learning progress is a key process in models of self-regulated learning and a key predictor of self-regulated learning and academic success. Judgments of learning (JOLs) are an established measure for assessing people's monitoring of learning and have been found to predict learners' subsequent performance as well as effort regulation. However, most studies have been conducted in laboratory settings, involving relatively artificial learning materials and low-stakes tests. We evaluate the predictive validity of JOLs for learning performance and effort regulation in an ecologically valid learning environment by requesting aggregate JOLs in an intelligent tutoring system. 90 German university students used an intelligent tutoring system that provided practice exercises for self-regulated preparation for a statistics exam over the course of a semester. Aggregate JOLs for each chapter of the statistics course were assessed once per week (279 assessments in total). Dependent variables were learning performance as well as absolute and relative learning effort for each chapter, derived from the intelligent tutoring system's log files. JOLs significantly predicted learning performance ( β = 0.20, p < .001) and effort regulation ( β absolute = −0.12, p < .001, β relative = −0.07, p = .002). The present research demonstrates that JOLs have predictive power in real-world learning. It thus bridges the gap between experimental cognitive research and applied educational research on metamemory and self-regulation. • Judgments of learning (JOLs) are predictions of one's own future performance. • Little is known about JOL accuracy in ecologically valid learning environments. • We examined JOL accuracy in an intelligent tutoring system used for exam preparation. • JOLs predicted effort regulation and performance during exam preparation.