Acoustic Correlates of Phases of Understanding in a Naturalistic Literary Task

Because large language models operate through language, they rely on self-reports to access the feeling of understanding, limiting their ability to support students’ learning. A previous study explored whether facial expressions could provide an alternative way of measuring this feeling across phases of understanding in an academic context, but it had limited success. To examine how effective other physiological channels may be in measuring the feeling of understanding, this study investigated whether acoustic feature patterns are associated with nascent understanding, misunderstanding, confusion, emergent understanding, deep understanding, and underconfidence as 198 participants completed a literary analysis task while their speech was recorded over Zoom. CatBoost and logistic regression models showed modest performance, indicating that phases of understanding were not reliably distinguishable at the population level. In contrast, within-person analyses revealed consistent differences between nascent and emergent understanding across several acoustic features, including pitch, jitter, shimmer, and spectral flux. The findings show that acoustic features of speech do not reliably distinguish phases of understanding at the population level in naturalistic academic contexts but do reflect consistent within-person differences between nascent and emergent understanding, highlighting both the potential and the limits of using speech as a physiological measure of the feeling of understanding and pointing to the need for alternative ways to operationalize this feeling as it unfolds across phases.