4D-IteraStereo: stereo matching via 4D cost volume aggregation and iterative optimization

Abstract Stereo matching is one of the most popular techniques in diverse applications such as localization, 3D scene reconstruction, and autonomous driving. In the pipeline of stereo matching, cost volume construction and aggregation are crucial components. Previous methods contain massive redundant features in 4D cost volume and require a great deal of 3D convolutions to aggregate cost, resulting in high memory requirements and computational costs. Moreover, current stereo matching methods neglect geometric information, leading to lower accuracy in textureless areas and reflective regions. In this paper, we propose 4D-IteraStereo aiming at optimizing cost volume construction and aggregation to improve matching accuracy in textureless areas. Specifically, 4D-IteraStereo builds a lightweight 4D cost volume by reducing redundant low-frequency features and selecting important channels. Then a series of parallel asymmetric and dilated convolutions are introduced to replace part of the 3D square convolution in the traditional aggregation module, which alleviates the computational burdens and better regularizes the 4D cost volume. After obtaining the coarse disparity map, the convolutional gated recurrent units (ConvGRUs) incorporate fusion cost volume to correct erroneous matching of the disparity map through state updates. The fusion volume additionally provides global and scene-prior knowledge to better address textureless areas and fine structures. Furthermore, extensive experiments are conducted on four datasets and real-world scenarios. Experimental results demonstrate that the proposed 4D-IteraStereo improves matching accuracy in fine structures and textureless regions.