Visual semantic encoding and identification of naturalistic movies via high-density diffuse optical tomography

Abstract Understanding how the brain represents meaning in real-world contexts is essential for fundamental neuroscience and clinical applications. Existing approaches rely on constrained scanning environments or conventional functional near-infrared spectroscopy (fNIRS), which is limited to sparse sampling or block-design paradigms. Here we show that high-density diffuse optical tomography (HD-DOT) supports naturalistic semantic encoding and decoding. We collect 3.5 hours of naturalistic movie data from six participants using stimuli labeled with 1708 categories. Encoding models predict voxel-level responses, yielding single semantic category maps consistent with prior fMRI studies. DOT responses capture sufficient semantic content to identify which clips participants viewed. A shared low-dimensional semantic space captures common semantic structure, and clustering analyses reveal interpretable higher-order dimensions such as animate and social agents. These findings show that DOT recovers distributed, high-dimensional semantic representations from naturalistic movies, opening new directions for naturalistic real-world neuroimaging in populations and scenarios difficult to access with fMRI.