Sophon: Byzantine-Robust Federated Learning via Dual Trust Mechanism

Federated Learning is a data privacy-protected distributed machine learning framework, but malicious clients can damage it. Byzantine-robust federated learning aims to learn an accurate global model despite the presence of malicious clients. Most current defenses assume that clients have identically and independently distributed (i.i.d.) data and thus cannot perform well in canonical non-i.i.d. scenarios. Several non-i.i.d. statistic-based defenses have been recently proposed to identify malicious clients through gradient statistics without any auxiliary techniques. They thus can only ensure robustness under certain attacks with specific characteristics. We propose Sophon, a comprehensive defense using auxiliary data to combat an arbitrary number of malicious clients in both i.i.d. and non-i.i.d. cases. Specifically, Sophon first normalizes received client gradients to reduce the dominance of malicious gradients. Then it introduces a dual trust mechanism to assign the aggregation weight for each gradient. The dual trust mechanism estimates the consistency-based and diversity-based trust scores of client gradients and integrates the two scores as the aggregation weight to effectively suppress the impact of malicious gradients. Extensive experimental results on three datasets from different domains, with diverse models and FL scenarios, show that Sophon is robust in maintaining the overall accuracy of the training model.