Visual processing of soft objects automatically activates physics-based representations in the human brain

When encountering soft objects, say a garment draping on a surface or a pillow being pressed, in the wrinkles and folds they make, we don't just see low-level properties such as edges, contours, or colors, but also seemingly higher level ones, such as mass, elasticity and stiffness. What neural and computational mechanisms underlie these percepts? Previously, using psychophysics and modeling, we found that human soft object perception is best explained by a model that incorporates "intuitive physics", as opposed to performance-matched alternatives that only consider pattern recognition (implemented as a CNN). Here, we hypothesize that, in the human brain, these intuitive physics-based representations (i) are computed spontaneously during visual processing, i.e., in the absence of physics-related tasks, (ii) occur in regions common with that of physical reasoning about rigid objects, and (iii) generalize across qualitatively different scene configurations. To address this, we used fMRI to scan participants (N=20) as they passively viewed animations of cloths at two stiffness levels (stiff and soft) undergoing naturalistic deformations in four different scene configurations (e.g., blowing in the wind, draping on an uneven surface). We identified each participant's regions of interest ("physics-ROI") using a previously validated localizer of physical inferences based on rigid objects. Univariate analysis showed that both physics-ROI and V1 were modulated by the soft vs. stiff cloths, but physics-ROI was modulated by this contrast to a significantly greater degree than V1. Moreover, multivariate analysis revealed successful cross-scene decoding of stiffness levels in physics-ROI (ACC=0.61). Notably, fine-grained rankings of cross-decoding accuracy across different scene configurations were well-captured by representations inferred in our physics-based computational model (Kendall’s τ=0.64) but not those in the performance-matched CNN (τ=-0.02). These results help reveal the implementation of physics-based representations of soft objects in the brain.