ASPipe: Adaptive Collaborative Pipeline Parallel Communication Acceleration Method

The pipeline parallel training of distributed deep neural networks (DNN) in edge computing faces the problem of cross phase communication delay. This article proposes the ASPipe method, which dynamically adjusts the compression rank of each stage through adaptive rank selection to reduce communication overhead, and combines multi-stage collaborative optimization strategies to improve training efficiency. Experimental results have shown that ASPipe significantly reduces communication latency in heterogeneous edge environments while ensuring model accuracy, resulting in a significant improvement in overall training efficiency.