Reduced Domain Multiplication for Deep Neural Network Acceleration

Demand for hardware resource reduction in deep neural networks has grown particularly as they grow deeper with more parameters. This research presents a novel concept known as the Reduced Domain multiplier, which leverages the resource-saving advantages of approximate hardware to evaluate tasks. The reduction in the multiplication domain is achieved with hardware implementation in sight. Canonical Signed Digit representations of the parameters and their extreme form of Signed Left Shift are proposed. We achieved accuracy comparable to that of the quantized networks for the CIFAR-10 and ImageNet-1K datasets using ResNet and ResNeXt architectures, as well as for the GLUE MRPC task on the BERT base model. This was accomplished with a 73% reduction in hardware resource usage per multiply operation on an FPGA implementation compared to an 8-bit quantized network. The resource savings can reach up to 87% by replacing the multiplier with a single left shifter, with accuracy reductions of 7.1%, 11.6%, and 6.8% for CIFAR10, ImageNet, and the MRPC task, respectively.