Asyn2F: An Asynchronous Federated Learning Framework With Bidirectional Model Aggregation

In federated learning, the models can be trained synchronously or asynchronously. Many existing works have focused on developing an aggregation method for the server to aggregate multiple local models into the global model with improved performance. They ignore the heterogeneity of the training workers, which causes the delay in the training of the local models, leading to the obsolete information issue. In this paper, we design and develop <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asyn2F</small>, an <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asyn</small>chronous <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</small>ederated learning <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</small>ramework with bidirectional model aggregation. By bidirectional aggregation, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asyn2F</small>, on one hand, allows the server to asynchronously aggregate multiple local models and generate a new global model. On the other hand, it allows the training workers to aggregate the new version of the global model into a local model, which is being optimized even in the middle of a training epoch. We develop <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asyn2F</small> considering various practical implementation requirements with geographically distributed and heterogeneous training workers. Extensive experiments with different datasets show that the models trained by <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asyn2F</small> achieve higher performance compared to the state-of-the-art techniques. The experiments also demonstrate the effectiveness, practicality, and scalability of <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asyn2F</small>, making it ready for practical deployment.