FedCSAC: Improving Accuracy and Privacy in Fully Decentralized Machine Learning with Clustered Sharding and Adaptive Differential Privacy Clipping

Decentralized Machine Learning (DML) enhances machine learning by improving scalability, adaptability, and privacy. Federated Learning (FL), a DML approach, boosts privacy by transmitting model gradients from local training to a central server instead of raw data. However, FL encounters challenges like potential privacy risk through gradient inference and communication bottlenecks. Several studies implement Differential Privacy (DP) to tackle these issues by adding random noise to gradients before aggregation. This approach safeguards sensitive information but can decrease model accuracy due to the introduced noise. Adaptive clipping helps by dynamically adjusting clipping bounds based on data, minimizing noise, and preserving data utility. However, in a decentralized system, noise application may be inconsistent across nodes, leading to learning inefficiencies. Our solution involves clustering, enabling local node collaboration and model aggregation to reduce DP noise impact and decentralize aggregation, minimizing bottlenecks caused by the central server. We also use data sharding to balance datasets across nodes, ensuring each processes representative data portions. This improves the signal-to-noise ratio, addresses data imbalance, and enhances accuracy. Experiments with MNIST and Fashion-MNIST datasets demonstrate that our method achieves better accuracy than conventional DP-SGD algorithms.