FPGA implementation of double-head SalsaNext: a CNN-based model for LiDAR point cloud segmentation

Abstract This study details the adaptation and deployment of a customized SalsaNext model for semantic segmentation of LiDAR point clouds on edge devices, benchmarked using the SemanticKITTI and Waymo Open datasets. We introduce an innovative multi-dataset training framework designed specifically for range image-based segmentation models. Central to this approach is our double-head SalsaNext model, which features two output heads to facilitate simultaneous training and inference on the Waymo and SemanticKITTI datasets. Following training, the model is streamlined by removing the head dedicated to Waymo, resulting in a compact, single-headed version optimized for SemanticKITTI. This simplified model is then quantized to employ fixed-point arithmetic, significantly enhancing computational efficiency and enabling real-time operation on the Xilinx Kria KV260 board. The quantization process markedly reduces resource consumption while preserving competitive accuracy. Our deployment on this low-power, FPGA-based platform underscores the potential of energy-efficient systems for advanced 3D semantic segmentation, with promising applications in autonomous systems and robotics. Experimental results validate the effectiveness of our training schema and the success of the optimized implementation of the double-head model on resource-constrained hardware.