LIFE: Leader-driven Hierarchical & Inclusive Federated Learning

Federated Learning (FL) is used to train machine learning models across distributed data and clients (devices). A key challenge in FL is addressing the complexities posed by heterogeneous clients with varying computational resources. While effective methods like knowledge distillation have been proposed to assist clients with limited resources, prevailing teacher-student models often focus solely on knowledge transfer, neglecting whether student clients can effectively and inclusively assimilate this knowledge. We introduce a leader-driven hierarchical distributed inclusive FL framework (LIFE) to address this challenge. LIFE engages hierarchical leaders across clients with various levels of computational capacities, aiming to optimize the performance of subordinate client models through a novel all-inclusive, leader-driven, level-by-level aggregation. Knowledge distillation is gradually achieved by fine-tuning subordinate models at all levels. Comprehensive experiments and comparisons with state-of-the-art methods across different tasks showcase that LIFE significantly enhances knowledge transfer across a variety of clients in an all-inclusive FL environment.