Incremental Sampling for Class Imbalanced Data Streams in Federated Learning

Federated learning is gaining attention for its ability to extract knowledge from distributed data streams without sharing client data. In this process, each client continuously collects samples from its local data stream for local training. The samples collected by clients at different points in time are only the tip of the iceberg of the overall dataset. The imbalance between these data chunks can be dynamically changing. At the same time, different imbalanced distributions may exist among clients, leading to spatial heterogeneity in data. To fill this research gap in federated learning, this paper presents the following innovation: 1) We identify nine characteristics of class imbalance in federated data streams and study their impact on global performance in a series of artificial scenarios; 2) We develop two new techniques, clustering-based incremental sampling and distance-based incremental sampling, to address the dynamic class imbalance in the federated data stream by retaining key samples locally at the client. Our incremental sampling method performs well and has set new records in real datasets regardless of the aggregation method.