When ML Training Cuts Through Congestion: Just-in-Time Gradient Compression via Packet Trimming

Distributed training of ML models generates significant network traffic when exchanging gradients and is sensitive to packet drops and retransmission caused by congestion when other traffic is sharing the network. While training processes can tolerate gradient compression to reduce traffic volume, it does not eliminate queue buildup in shallow-buffer switches, causing high latency and extended training time.