High-dimensional latent manifolds as predictors of individual differences in naturalistic movie viewing

The human visual system is adept at processing complex, high-dimensional sensory data. A prominent theory proposes that the visual system accomplishes this by transforming high-dimensional sensory inputs into simpler, low-dimensional representations. However, recent theoretical and empirical work suggests that the dimensionality of visual cortical representations may be more extensive than previously thought. We hypothesize that even low-variance dimensions in cortical population activity are critical to human vision and that individual differences in visual experience are captured by these high-dimensional codes. To investigate this possibility, we used a recent method, known as cross-decomposition, to identify the shared high-dimensional signal between pairs of individuals. We applied this method to publicly available fMRI data collected from forty participants while they viewed four short movies. We first computed cross-validated covariance spectra between subject pairs, creating a matrix that reflects individual differences in the high-dimensional latent space. We then measured the reliability of these individual difference matrices by computing their correlation across pairs of movies. Our analysis revealed a long-tailed spectrum of reliable, low-variance dimensions shared among individuals in the ventral visual stream, and we found that these high-dimensional signals were highly consistent for the same subject pairs across different movies. These findings suggest that there are rich and reliable individual differences in the high-dimensional representational manifolds that underlie naturalistic visual experience.