An Energy-Area-Efficient 3D Interleaved-Ring Accelerator With INT/FP Pipelined PE Array and 3D-SRAM Cube for On-Device CNN Training and DM Inference

Deep neural networks (DNNs) have demonstrated exceptional performance in image-related artificial intelligence (AI) applications. However, the inference of generative models, such as Diffusion Models (DMs), and the training of convolutional neural networks (CNNs) are computationally intensive tasks, requiring extensive floating-point (FP) operations to maintain accuracy and high-quality results. These tasks are also memory-bound, generating large intermediate data that result in significant external memory access (EMA), thus complicating the deployment of image-related DNNs on edge devices. While 3D-stacked SRAM with through-silicon via (TSV) technology offers promising solutions to alleviate EMA, the complexities of 3D interconnect architectures can introduce substantial overhead in intra-chip communication, potentially degrading overall efficiency. In this paper, we propose a flexible 3D interconnect architecture, termed the 3D interleaved-ring, which utilizes multiple 3D interleaved rings to connect the pipelined integer (INT) and floating-point (FP) processing element (PE) arrays with a 3D-SRAM cube, effectively mitigating the overhead caused by 3D interconnection. We design <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3\times 3$</tex-math> </inline-formula> micro-routers with dedicated channels and embedded adders for the proposed 3D interconnection architecture. This 3D ring-based architecture reduces the power consumption of on-chip data movement by a factor of 4.5 and decreases the number of required TSVs by a factor of 4.2, compared to conventional 3D mesh-based interconnects. Additionally, we introduce efficient mixed-bit precision dataflows that incorporates dynamic workload distribution to optimize data reuse, reduce bandwidth demands on the 3D-SRAM cube, and improve PE utilization. The proposed work achieves over 90% PE utilization and reduces DRAM accesses by more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$7\times $</tex-math> </inline-formula> across various DNN models. Overall, the proposed 3D accelerator improves energy and area efficiency by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8.4\times $</tex-math> </inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.4\times $</tex-math> </inline-formula>, respectively, compared to state-of-the-art DNN accelerators and processors.