Master-Device-Coordinated Optimization for Distributed CNN Inference at the Edge

The application and services of artificial intelligence (particularly deep learning) have experienced explosive growth. However, cloud computing service architectures face limitations such as efficiency and latency when supporting deep learning inference tasks, which constrain the performance of these technologies in real-world scenarios. Thus, researchers have begun to explore offloading inference tasks to edge devices. Yet, edge devices typically suffer from limited computational resources and memory, making it difficult to meet the stringent requirements of low-latency and high-efficiency services. To address these issues, we propose EdgeMDC, a distributed inference framework that intelligently partitions intermediate feature maps and assigns them to edge devices within a cluster. Additionally, to optimize the transmission of padding data, a quantization-based compression mechanism coordinated by a master device is introduced. Experiments show that the proposed scheme reduces communication overhead and improves the inference speedup by 16.8% compared to state-ofthe-art methods.