eTSDF: An Energy-Efficient Accelerator for TSDF-Based 3D Scene Reconstruction in FPGAs

Three-dimensional (3D) scene reconstruction from the depth sensor is one of the fundamental research problems in many emerging applications such as robotic and virtual reality. However, due to the high computational complexity of 3D scene reconstruction processes, it is very difficult to achieve real-time implementation on low-power platforms. In this paper, we propose eTSDF, an energy- and memory-efficient hardware architecture for real-time truncated signed distance field (TSDF)-based 3D scene reconstruction system by accelerating the entire process stages on field-programmable gate array (FPGA) platform. To achieve memory-efficiency, we propose an adaptive TSDF integration solution, named eTSDF, which leverages the spatial locality of 3D volumetric representations and dynamically conducts integration process on sub-volumes, rather than on the entire volumes. In eTSDF, we design a set of dedicated, well optimized accelerators to achieve real-time implementations for the processes of adaptive sub-volume construction, coordinate transformation and TSDF integration. Evaluation results demonstrate that the eTSDF accelerator achieves up to 1.84× speedup and 101.23× energy efficiency improvement compared to the Intel i9-12900K CPU.