A Normative Account of 3D Motion Perception from Natural Scenes

Two monks were watching a banner fluttering in the wind. One said, “The wind is moving.” The other said, “The banner is moving.” They argued back and forth, unable to agree. Huineng approached and said, “It is neither the wind that moves, nor the banner that moves. It is your mind that moves.” —Platform Sutra Perception is not a passive reflection of reality, but an active inference. Our visual system receives 2D images on the retinae and must infer the underlying 3D layout of the environment. This inference is challenging because the same retinal image can be produced by many distinct 3D structures. To resolve this ambiguity, the brain uses multiple depth cues, such as binocular disparity, motion parallax, occlusion, blur, and perspective distortion (Howard and Rogers, 2008). The neural mechanisms underlying some of these cues are well studied, while others remain less clear (Howard and Rogers, 2008; Anzai and DeAngelis, 2010). Beyond estimating static 3D structures, the ability to detect movement in the environment is also critical for survival. Many species rely on motion perception to navigate their environment, avoid predators, and capture prey. Motion in the frontoparallel plane—i.e., 2D motion—has been extensively studied (Park and Tadin, 2018). These studies have revealed that in the primate brain, object motion processing begins in the primary visual cortex (V1) and proceeds along the dorsal visual pathway, including the middle temporal (MT) and medial superior temporal (MST) areas (Park and Tadin, 2018). Less is known about how the visual system processes 3D motion (or motion-in-depth, MID), however. A particular challenge arises when objects move toward or away from us (or when we move in depth relative to an object). In such cases, our brain must integrate both motion … Correspondence should be addressed to Zhe-Xin Xu at brian_xu{at}hms.harvard.edu.