A Two-Stage Approximation Methodology for Efficient DNN Hardware Implementation

Deploying high-performance Deep Neural Networks (DNNs) on embedded hardware platforms, such as FPGAs, not only promises low latency and high energy-efficiency implementations, but also poses significant challenges due to the resource constraints of such systems. Recent efforts have focused on approximation techniques at the algorithmic level of DNNs to reduce resource requirements, such as network pruning and quantization of weights and activations. Other approximation techniques that approximate register transfer level (RTL) designs by substituting single components with approximated ones are not yet applied to DNNs, mainly because these techniques do not scale well to the typically huge RTL representations of DNNs.This paper introduces the idea of a two-stage approximation methodology for DNNs that combines algorithmic level with RTL approximation to achieve reduced resource requirements at acceptable accuracy. As a novel instance of that idea, we leverage the LogicNets approach that performs a low-bit quantization to map DNN neurons to LUT representations and the CIRCA framework that provides a search-based approximation flow for RTL designs. To balance hardware efficiency and accuracy, and to achieve acceptable runtimes, we limit the RTL approximation to a subset of neurons identified as resilient using a systematic sampling approach. Experimental results demonstrate the potential of the methodology with up to 15.20% further decrease in resource utilization with a mere 2.15% accuracy drop.