P4Com: Implementing P4-Based In-Network Data Reduction for Geo-Distributed Machine Learning

Geo-Distributed Machine Learning (GDML) aims to enable datacenters to collaborate in training large-scale models. These datacenters are located in different geographic regions. However, the limited Wide Area Network (WAN) bandwidth resources restrict the performance of GDML systems. Existing solutions compromise model accuracy, while in-network computing-based approaches suffer from issues such as complex processing logic, high system implementation overhead, and a high probability of gradient aggregation conflicts. Therefore, this paper proposes a lightweight in-network data reduction mechanism, P4Com, based on the P4 programmable data plane. First, we design the network-layer identifier to represent gradient aggregation tasks. Then, a register conflict avoidance mechanism is proposed to improve register utilization efficiency. Building upon this, we design a lightweight data plane using the protocol-independent P4 language to support line-rate in-network gradient aggregation. Finally, A prototype system is built for validation, and experimental results show that P4Com significantly reduces the runtime latency of GDML systems.