Multi-machine Adaptive Freezing algorithm for Enhanced Pipeline Parallelism Training in 6G Networks

As the scale of deep learning models continues to expand, computational cost has become a major challenge. Model parallelism techniques, especially pipeline parallelism, have been shown to effectively reduce training time and improve device utilization. However, the high financial cost of traditional single-machine multi-GPU (multi-Graphics Processing Unit) training environments has hindered the progress of many research efforts. Against this backdrop, this paper explores the feasibility of large-scale distributed model training in heterogeneous, multi-machine environments in 6G networks. To this end, we propose a novel training method called Multi-machine Adaptive Freezing (MmAF). The method combines pipeline parallelism with dynamic layer freezing strategies, multi-machine environments, significantly reducing both training time and communication overhead. A multi-machine pipeline parallelism testbed was developed. Experimental results demonstrate that, compared with full finetuning training, MmAF reduces training time by 15.92% while ensuring no degradation in performance.