CAFNet: Efficient Cost Aggregation via Conv-Atten Fusion for Real-Time Stereo Matching Toward Autonomous Driving

Stereo matching that provides dense 3D information is crucial in the perception of autonomous driving. Most state-of-the-art stereo matching networks pursue higher prediction accuracy relying on the strong regularization of 3D convolutions, but at the cost of speed. The computational complexity and memory consumption brought by 3D convolutions hinder the application of stereo matching in real-time scenarios. Thus, in this paper, we propose a two-stages stereo matching network with lightweight shallow 3D cost aggregation in the first stage and 2D cost aggregation in the second stage. We propose conv-atten fusion (CAF) block to efficiently explore positional and contextual information over a large receptive field. On the basis of CAF, a learnable regularized upsampling (LRU) is proposed to enable 3D aggregation on low-size cost volume to significantly cut down resource consumption. LRU provides a learnable way to upsample the regularized cost volume with much less ambiguities. By incorporating CAF into a simple 2D cost aggregation, a non-local aggregation is constructed to effectively yet efficiently regularize the cost volume in the second stage to obtain reliable matching proposals. By evaluating on KITTI benchmarks, the accuracy of CAFNet surpasses all published real-time results, and its comprehensive performance is outstanding among all published state-of-the-art results in terms of the proposed quality factor.