Hardware-Aware Automated Quantization for Balancing Latency and Accuracy

The Hardware-aware Automated Quantization with Mixed Precision (HAQ) method faces limitations due to the reinforcement learning cycle's inadequate consideration of hardware-induced latency, which hinders the attainment of Pareto optimization. This study conceptualizes hardware-perceptive quantization as a multi-objective reward problem, introducing a latency factor into the equation. Given the disparate influences of latency and accuracy on the reward structure, a series of warm-up rounds are integrated prior to the reinforcement learning process to achieve a harmonious balance between these parameters. Empirical evaluations conducted on the Zynq-7020 demonstrate a considerable reduction in latency while preserving comparable accuracy to the HAQ method, with a fixed cost incurred across a limited number of iterations.