A High-Performance Approximate Softmax Architecture

Deep learning is rapidly advancing. However, the increasing computational demands present challenges for high- efficient hardware implementations. A novel softmax architecture based on approximate computing is proposed. The exponential operation in softmax is transformed from base e to base 2. Additionally, Goldschmidt approximate algorithm is applied to simplify division operation. Through segmented approximate radix-4 Booth multipliers, the area and power consumption are effectively reduced while maintaining high accuracy. The proposed design is synthesized under SMIC 28-nm CMOS technology and tested on Vision Transformer classifying the ImageNet dataset. Compared with the state of the art, proposed approximate softmax achieves up to 64.1 times area efficiency while maintaining high precision.