Investigation of the Effects of Variable Accuracy Approximate Units in Deep Neural Network Accelerators

Deep Neural Network accelerators are now part of most SoCs for desktops and mobile phones. The computational elements in these accelerators mainly comprises of MAC units distributed across the design, and they operate in parallel to generate the inference. To reduce the footprint of the digital implementations, different approaches have been used such as data quantization or using approximate arithmetic units. In this work, we have investigated the effect of varying the level of approximation possible in the multiplier of the MAC unit, in the different layers of the DNN architecture. A distinct pattern was observed, DNN architectures are more tolerant to high levels of approximation in the latter layers, while the initial layers require high accuracy approximate units. This allows the use of variable precision approximate units depending on the layers, thereby improving the gains possible with these approximate computational units.