Adaptive Fit Fraction Based on Model Performance Evolution in Federated Learning

Federated Learning (FL) allows distributed training over data in clients devices, where distributed local models are aggregated in a central server to build the so-called global model. Selecting clients to participate in the distributed training process can help in improving training efficiency and computing performance by reducing training rounds and data transfers through the network. Since selecting involves creating a subset of clients who will participate in training, it is necessary to define the size of that subset (fit fraction) appropriately. This definition is an open problem. Using a constant fit fraction throughout the training is common, and its definition requires prior knowledge of the context where the training takes place. Proposals that modify it make aggressive changes, which can lead to over-usage of training resources or under-usage of training data. The proposed solution, Adaptive Fit Fraction (AFF), tackles these issues by modifying the fit fraction, considering the changes in training performance over time. It uses an observation window and linear regression on the model’s performance in this window to identify the trend and intensity of training evolution. Based on these metrics, AFF defines the magnitude of the change on the following observation window’s fit fraction value. The results demonstrated that the proposal does not negatively impact the model’s performance and that it reduces costs by requiring less client participation in more stable scenarios.