DNPS: A Robust Aggregation Method for Heterogeneous Distributed Learning Based on Gradient Direction and Norm Probability Screening

Heterogeneous distributed machine learning systems are vulnerable to Byzantine attacks, where malicious worker nodes disrupt global model convergence by submitting incorrect model updates, leading to degraded system performance. Current aggregation methods that filter based on gradient norm or direction often underperform in heterogeneous environments and struggle to defend effectively against Byzantine attacks. To address this issue, we propose the Direction-Norm Probability Screening (DNPS) algorithm-a novel approach that harmonizes gradient norm and direction to blunt these attacks. DNPS integrates gradient norm and directional information to establish a probability screening mechanism for identifying and filtering Byzantine worker nodes, thereby fortifying the system's defen-sive capabilities and enhancing its robustness against Byzantine attacks. Experimental results show the DNPS algorithm provides more effective protection under both non-attack and Byzantine attack scenarios than existing aggregation methods.