RISCSparse: Point Cloud Inference Engine on RISC-V Processor

Machine learning on point clouds is increasingly accessible at the edge, notably in applications such as autonomous driving. However, the sparse and irregular nature of point clouds presents significant latency challenges on general-purpose hardware. RISC-V, with its evolving ecosystem, offers a promising platform for embedding intelligence at the edge due to its full-stack scalability. This paper focuses on the advanced point cloud operation known as submanifold convolution (SC), deploying submanifold sparse convolutional networks (SSCNs) on a RISC-V System-on-Chip (SoC) designed within the Chipyard framework. We address three critical bottlenecks of SSCNs- Rule Map Construction (Mapping), Gather-MatMul-Scatter (GMS), and uncombined operation - to meet the real-time inference requirement for the on-chip implementation. By leveraging the RISC-V Vector extension and Gemmini, an open-source full-stack DNN accelerator generator, we vectorize the Mapping process, offload GEMM-related operations to the Gemmini Systolic Array, and cooperatively use the Systolic Array and vector processing units to reduce the memory footprint. Our evaluations show that the RISC-V-based SSCNs implementation achieves an average of 11.73× and 13.1× overall speedups with a small workload compared to TorchSparse on Edge-CPU, a state-of-the-art point cloud inference engine, for 3D segmentation and detection tasks, respectively. When contrasting with TorchSparse on an Edge-GPU, our implementation still delivers a notable improvement, with average speedups of 1.63× for 3D segmentation and 1.07× for detection tasks.