Efficient and Flexible Deep Learning Processor for Embedded Devices With 99.8% PE Utilization Under Low Silicon Cost

The flexibility and efficiency of accelerator hinder the realization of DNN (Deep Neural Network) on embedded devices. How to design efficient and flexible deep learning processors to meet low-power requirements while supporting various iteratively updated deep neural networks, which is a major challenge in the current field. Moreover, the significant power consumption and latency generated by data access between on-chip and off-chip memory have been the bottleneck of DNN accelerator development. We hence design the flexible and efficient deep learning processor (DLP) based on our designed specific neural network instruction set for DNN inference, which can be used to flexibly accelerate the inference of iteratively updated neural networks. Our design includes the design of hardware architecture, neural network acceleration instruction set, efficient SIMD control path and data path, which can be used to accelerate the execution of the DNN programs, and support hiding the data transfer time between on-chip and off-chip memory. In addition, we propose a parallel and conflict-free data access scheme to reduce data access overhead in memory system. Moreover, we also propose a scheduling framework to improve performance and minimize power consumption under the hardware resource constraints. The experimental results indicate that our design’s power consumption efficiency is 75.3 TOPS/W at TSMC 65 nm process. Our power efficiency is 7.6, 5.5 and 4 times higher than the state-of-art DNN accelerators respectively.