Low-Power Register File for Tensor Cores

The introduction of tensor core (TC) in NVIDIA GPUs has accelerated neural network computations. A TC is a set of storage and arithmetic units dedicated to handling matrix-multiply-and-accumulate (MMA) operations. While TC reduces the runtime of convolutional neural networks (CNNs), it increases power consumption. In particular, register files within TCs consume a significant fraction of leakage power as GPU designers steadily have increased the size of register files to boost performance. In this work, we propose a value-based approach to reduce leakage power in register files. We observe that register bits exhibit a strong bias towards zeros. We exploit this bit-level bias property and propose a low-power SRAM (LPS) cell that draws significantly less leakage current than regular SRAM cells. In the preferred state, the leakage power is smaller by as much as approximately 49×. We also propose LPS+ which increases the sparsity rate in SRAM cells by selectively inverting register bits. To reduce leakage power further, we propose precision-aware LPS+ (PLSP+) which exploits the error resiliency property of CNNs and drops low-order bits in network values. We evaluate our proposed techniques using state-of-the-art CNNs and show that leakage power is reduced by 77.3% with a negligible impact on accuracy.