Efficient Hierarchical Quantization for Heterogeneous Devices in Cloud–Edge–Device Architecture

Cloud-based quantization is a key technique for deploying deep neural networks on resource-constrained devices. However, the growing number of heterogeneous devices has placed an increasing burden on the cloud server. It is necessary to handle the high costs of quantizing original models to multiple bitwidths on the cloud server. Therefore, we propose an efficient hierarchical quantization method (HQAQ), which transforms classical cloud-based quantization to two-phase quantization: cloud-edge post-training quantization and edge-device hybrid re-quantization. HQAQ first quantizes original models on the cloud server and distributes the quantized model to edge servers. Then the edge servers re-quantize the quantized model to respond to heterogeneous devices’ requests. To minimize the computational costs on edge servers, edge-device hybrid re-quantization integrates top-bitwidth quantization-aware training with cross-bitwidth quantization points transfer. Quantization points transfer employs clustering to discover the distribution of the quantization points with higher bit-width, which helps quantize the models of lower-bitwidth models. The experimental results on image classification and time series prediction tasks demonstrate that the proposed method improves communication efficiency while maintaining model performance.