Low-Error ASIC Implementation of SoftMax Activation Function for Deep Neural Networks

Deep Neural Networks (DNNs) are one of the prominent and state-of-the-art methods for implementing artificial intelligence algorithms. A noticeable amount of effort has been put into hardware acceleration of DNN, with significantly less attention given to the SoftMax layer, which uses expensive hardware for exponentiation and division. The SoftMax function must be implemented as accurately as possible since it is crucial in multiclass classification training and inference tasks. This paper describes an efficient implementation of the SoftMax function for error-sensitive application of DNNs. We propose a Lookup Table (LUT) based on linear polynomial approximation for exponential and log units. The proposed architecture is modeled in Verilog and synthesized to gate-level netlist in GPDK180[Formula: see text]nm using Cadence[Formula: see text]. Results after the placement and routing stage show a delay of 7.15[Formula: see text]ns while consuming 5.3 mW of total power when operated at 100[Formula: see text]MHz. Multiclass classification is coded in Python with the proposed SoftMax layer to evaluate the accuracy in real-time applications. The results show that the neural network trained with the proposed SoftMax reaches almost the same accuracy as traditional SoftMax of 86.76%. This proves that low-form factor platforms can implement the same accuracy achieved with the proposed SoftMax function with less power.