A 13-34 TOPS/W Edge-AI Processor Featuring Booth-Value-Confined Accelerator, Near-Memory Computing, and Contiguity-Aware Mapping

Recently, edge artificial intelligence (AI) devices, particularly those with in-memory-computing (IMC)/near-memory-computing (NMC) architecture, have demonstrated superior power efficiency compared with conventional von-Neumann architecture [1]–[7]. To further improve the power efficiency, approximate computing techniques have been investigated [8], which inevitably incur accuracy loss as such approximations cannot be evaluated during the NN training stage on GPU (Fig. 1). In addition, despite the exceptionally high power efficiency at the macro-level, the often-overlooked resource consumption for inter-/intra-layer data alignment and reshape can actually lead to substantial overhead of power consumption and latency. This is caused by 1) inefficient memory bandwidth utilization due to irregular or unbalanced read/write patterns (e.g., [3] [4]), and 2) severe memory space discontinuity due to inefficient neural network (NN) mapping strategy (e.g., [6] [7]), as shown in Fig. 1. Hence, when considering the end-to-end NN execution, aspects of off-chip memory traffic and network mapping strategy require further investigations at the architectural level.