A 4.69-TOPS/W Training, 2.34-μJ/Image Inference On-Chip Training Accelerator With Inference-Compatible Backpropagation and Design Space Exploration in 28-nm CMOS

On-chip training (OCT) accelerators improve personalized recognition accuracy while ensuring user privacy. However, previous OCT accelerators often required significant additional hardware costs to support retraining, even though inference is the primary use case. We propose an inference-pattern-compatible backpropagation (BP) circuit, which enables the training process to reuse inference hardware. To achieve high energy efficiency, we utilize three hardware-friendly optimization methods that significantly reduce redundant computation and external memory access (EMA). Additionally, we propose a design space exploration (DSE) to search for the optimal hardware configurations, which improves system performance while reducing the design time cost. Fabricated in a 28-nm CMOS process, this single-core OCT chip is able to train all the layers of a neural network (NN), achieving a peak training efficiency of 4.69 Tera operations per second per watt (TOPS/W). It also achieves the lowest inference energy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.34~{\mu }$ </tex-math></inline-formula>J/inf/image under a core voltage of 0.48 V and 40 MHz.