One at a time in the mind’s eye: Serial versus parallel mental simulation of moving objects

Our everyday environment is highly dynamic, and this creates great challenges for any cognitive system. Decades of research using the Multiple Object Tracking paradigm has shown that when people view moving objects, they can only track a handful of them at a time. But people’s real-world tracking extends beyond direct perception: we can also keep track of occluded bodies, as well as imagine the yet-unseen future paths of objects (e.g., in order to estimate where a thrown ball will land). In this work, we examined the capacity limitations of tracking items in imagination, rather than perception. Across 4 experiments, participants (N=136 total) watched short 2D animations of balls moving under gravity. Animations paused mid-motion, and participants were asked to continue the scene in their mind’s eye, and indicate when each ball hits the ground in their imagination. Responses were compared with the true impact time, which was extracted from simulated physics. With a single ball, people’s imagination-based responses closely matched the true impact time (Exp. 1). However, once another ball was introduced, performance was significantly altered (Exp. 2), and followed the predictions of a computational model of serial simulation, which only moves a single object forward at a time. The serial pattern was not due to response requirements (Exp. 3), and was reduced, but not fully eliminated, by introducing extremely strong grouping cues that even allowed relying on heuristics (Exp. 4). Together, our results show that tracking objects in imagination is, at least under certain conditions, a serial process that operates on a single-object basis. More broadly, examining ‘imagination tracking’ highlights previously unexplored capacity limits in mental simulation, calling for an updating of current models of how humans make intuitive predictions of physical outcomes.