Resource-efficient Federated Learning for Large-scale Model Training

Federated Learning (FL) effectively strengthens the protection of user privacy data, but it is not suitable for large-scale model learning due to prohibitive communication, storage, and training costs. Secure Federated Submodel Learning (SFSL) is an innovative framework proposed to enhance FL by having users train only a portion of the complete model, known as a submodel. This approach can effectively reduce download overhead. However, SFSL introduces new challenges, such as extremely high training time and privacy leakage, as it relies on information about submodel selection. To solve these challenges, we propose Parallel Secure Federated Submodel Learning (P-SFSL). P-SFSL adopts a parallel structure for training, significantly reducing training time and enabling lightweight devices to participate in the process. We evaluate P-SFSL on Alibaba's recommendation system, and P-SFSL outperforms SFSL with 80% training time reducing and better privacy protection.