A Real-Time Optical-Flow-based SLAM FPGA Accelerator with Inter-Frame Similarity Exploitation and Correlation-Guided Mixed-Precision Flow Update

Recently, the increasing demands of scene understanding and precise spatial mapping have driven Simultaneous Localization and Mapping (SLAM) more crucial in robotics for estimating the trajectory and the structure of the environment [1–2]. Previous feature-based SLAM processors [3–5] retrieve pose and 3D maps by tracking limited feature points $({\lt}5 k)$, making them vulnerable to changing environments and illumination. In addition, feature-based SLAM cannot directly construct dense maps, impairing their applicability in advanced robotics. In contrast, optical-flow-based SLAM (OpFSLAM) [6] fully utilizes pixel-level information to provide continuous motion estimations and generate dense map representations, meeting the diverse requirements of emerging mobile robots. An overview of OpF-SLAM is shown in Fig. 1, which involves extracting per-pixel feature and context information, constructing correlation volume (Corr) for all pixel pairs, iteratively updating optical flow, estimating the camera trajectory, and building environment maps.However, OpF-SLAM encounters several challenges in achieving real-time (>30fps) tracking, as shown in Fig. 1: (1) The data dependency between feature extraction and correlation construction causes latency overhead. (2) Iterative optical flow updates require an excessive computational amount, accounting for 62.19% of the total computation.