Decoding predicted future states from the brain’s ‘physics engine’

Successful engagement with the physical world requires rapid online prediction, from swerving to avoid a collision to returning a ping-pong serve. Here we test the hypothesis that physical prediction is implemented in a set of parietal and frontal regions (aka the "hypothesized Physics Network '' or hPN) that model the structure of the relevant scene and run forward simulations to predict future states. For physical scene understanding and prediction, contact relationships between objects such as support, containment, and attachment are critical because they constrain an object's fate: if a container moves, so does its containee. In Experiment 1, participants (N = 14) were scanned with fMRI while viewing short videos (~3s) depicting contact (contain, support, attach) and non-contact events. MVPA revealed scenario-invariant decoding of the presence versus absence of a contact relationship that was significant in the hPN but not in the ventral pathway. Experiment 2 tested whether the hPN also carries information about predicted future contact events, as expected if the hPN is engaged in forward simulation. Indeed, the voxel response patterns in hPN distinguishing between perceived contact and no-contact events were similar even for predicted events where contact was predictable but not shown. This prediction of future contact events, which generalized across objects and scenarios, was found even though participants were performing an unrelated one-back task, and was detected only in the hPN, not the ventral visual pathway. In both experiments, the key results were absent in the primary visual cortex, arguing against low-level visual feature confounds accounting for these findings. Thus, we find that the hPN both (a) encodes physical relationships between objects in a scene, and (b) predicts future states of the world, as expected if this network serves as the brain’s ‘Physics Engine’.