TCANet: Three-dimensional cross-attention mechanism for stereo-matching

Abstract Effective disparity estimation is a current hotspot in stereo vision research, and cost aggregation is an important part of disparity prediction. How to perform more effective cost aggregation is the core step to improve the accuracy of disparity prediction. In previous studies, 3DCNN with a stacked hourglass structure is often used for cost aggregation. In this research, we propose an effective 3D cross-attention stereo network that utilizes the attention mechanism to obtain contextual information for cost aggregation in a more efficient way. Specifically, the 3D cross-attention module in TCANet acquires the geometric information of all pixels on the 3D cross path. By repeating this operation twice, each pixel can eventually obtain global dependencies from all other pixels. Using the 3D cross-attention module in a stacked hourglass-structured 3DCNN only increases the number of parameters by a very small amount, which can effectively improve the performance of the model. Experimental results show that TCANet performs well on virtual dataset Scene Flow and realistic KITTI datasets.