Predictive Looking and Predictive Looking Errors in Everyday Activities

Where people look in pictures and movies has been shown to be based not only on the most salient point in the current scene, but also on predictions of what is going to happen next. The accuracy of these predictions fluctuates during movie watching. Some theories of event comprehension propose that spikes in prediction error can trigger working memory updating and the segmentation of ongoing experience into meaningful events. One previous study of predictive looking found evidence for this proposal (Eisenberg et al., 2018, CR:PI), but the paradigm used in that study could only obtain predictions intermittently, because it analyzed predictive looking to objects that an actor was about to contact. Here, we developed a continuous measure of prediction error by modeling predictive looking towards the actor's hands, and we operationalized this prediction error as the residuals from the predictive looking model. Viewers’ gaze was tracked while they watched movies of everyday activities, and mixed-effects models were used to predict the actor’ hand positions from viewers’ previous gaze location. Stepwise model comparison indicated that viewers look predictively as current gaze position accounts for hand location as far as 9 seconds in the future. We compared the time course of gaze predictions with that of predictions generated from a computational model of event comprehension and found that gaze predictions showed higher error at moments when the computational model had higher errors. Furthermore, spikes in gaze prediction error were predictive of increases in event segmentation in a separate group of viewers. These results support proposals that event segmentation is driven by spikes in prediction error, and this method promises to give a general approach for measuring ongoing prediction error noninvasively.