VEGA: Implementing a Versatile and Efficient Deep Learning Processor with Graph-Based ALU

As neural networks advance, the diversity and latency proportion of non-matrix-multiplication operators (NMO) are on the rise. Providing a versatile and efficient acceleration for this intricate set of NMOs poses great challenges in hardware design. In this work, we analyze the algorithmic structure of NMOs and propose graph-based ALU (GALU) to improve efficiency. The key idea is to organize functional units into a dataflow graph with a configurable interconnection, which reduces computation time and memory access. Further, we provide architectural support to integrate GALU into a multi-thread processor called VEGA, which supports various NMO structures. At the hardware level, we devise swift interconnection reconfiguration (SIR) for GALU to reduce the latency caused by reconfiguration. We also design a fine-grained instruction scheduler to fully utilize SIR. At the software level, a three-stage compilation framework is developed to enhance the usability. Experiments demonstrate that GALU achieves a 2.27x speedup with only an 18.11 % increase in area overhead. Compared with NVIDIA Jetson Orin, the VEGA prototype achieves a 3.84x speedup on typical NMOs and achieves a 2.03 x end-to-end speedup at the network level.