Hawkeye: A Point Cloud Neural Network Processor With Virtual Pillar and Quadtree-Based Workload Management for Real-Time Outdoor BEV Detection

Large-scale 3-D processing using the point cloud neural network (PNN) has become essential for applications such as an autonomous driving system. Among the various methods, pillar-based PNN is widely utilized for processing large-scale outdoor environments. However, due to the irregular and sparse nature of the point cloud compared to the regular and dense image, achieving high efficiency has been challenging in previous works. Inspired by the LiDAR mechanism that generates point clouds, this article proposes Hawkeye, a large-scale bird’s-eye-view (BEV) detection processor designed for end-to-end processing of pillar-based PNN. Hawkeye incorporates three key features: 1) a feature encoding core (FEC) that introduces virtual pillars (VPs), dynamically generating temporal pillars to facilitate continuous processing; 2) a map management core (MMC) that utilizes a quadtree-based workload management system, enabling region-of-interest (ROI)-based skipping to optimize computational efficiency; and 3) a low-precision neural engine (LPNE) that employs a bit-slice compute architecture with sign-magnitude (SM) representation, enhancing slice-level skipping (SLS) capabilities. Fabricated in Samsung’s 28-nm CMOS technology, Hawkeye achieves a performance of 44.2 frames per second (FPS) at 320 MHz on the PointPillars with the KITTI dataset. Furthermore, with the aid of ROI-based skipping and SM-based SLS, Hawkeye demonstrates an effective energy efficiency of 38.5 TOPS/W under 0.74 V and 40 MHz.