A real-time binocular stereo vision method based on FPGA

To meet the environmental perception requirements in fields such as robotics and autonomous driving, this paper proposes an enhanced binocular stereo vision model using a visible light camera and introduces a novel FPGA architecture for its implementation. The classical binocular stereo vision method integrates convolutional neural networks (CNNs), which significantly reduce matching errors. However, this improvement comes at the cost of increased matching time, which, while lowering deployment costs and improving environmental information acquisition, results in practical applications that fall short when compared to methods like LiDAR.To address this issue, we present a real-time stereo matching network that incorporates semantic information. By employing a lightweight design, we substantially reduce the number of parameters while maintaining high accuracy in matching. Additionally, we integrate semantic segmentation branches that allow the network to extract semantic information about the surrounding environment. This not only optimizes stereo matching through semantic features but also enhances environmental awareness technology. Moreover, we design a series of parallel structures to ensure excellent performance on edge computing platforms, such as mobile devices, with the capability for FPGA deployment and acceleration. Finally, we validate our method on the Xilinx FPGA ZYNQ 7010 platform, demonstrating excellent real-time performance and practical results.