Logarithmic Format for Quantization on Forward Passes with High Accuracy

With the growing demand for Artificial Intelligence (AI) products, the need for efficient tools and methodologies to minimize processing overhead in large models has never been greater. As AI applications extend their reach into high-stakes scenarios, the trade-off between accuracy and performance metrics is inevitable. In our paper, we propose the utilization of the Logarithmic Number System (LNS) to improve accuracy while retaining the performance advantages of rapid operations that are crucial to AI applications. Our proposed approach involves defining a logarithmic format to represent values in models. We propose an approach to achieve accurate and rapid inferences by using quantization and implementing fundamental operations like multiplication, division, addition, subtraction, and an approximation of the dot product. Our experiments showed that LNS can have significant speedups for dot product calculations, especially for large vectors. However, the results indicated a trade-off with reduced precision, as outputs are heavily quantized. By tuning parameters such as <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\sigma$</tex>, we were able to reduce some accuracy losses while maintaining efficiency. These findings demonstrate the potential of LNS to enhance computational performance while balancing accuracy for performance-critical applications.