Generative Learning-Based Personalized Federated Learning for Metaverse Data Security: Proposing Two New Frameworks

The vision of the metaverse encompasses the seamless integration of the real world and the virtual world. Users engage in the metaverse by uploading large amounts of heterogeneous personal data to servers. Such an approach inevitably leads to privacy breaches and significant communication overheads. Federated learning (FL) is a distributed machine learning paradigm that can effectively address both of these issues. In addition, customizing personalized models for each client can effectively mitigate the impact of data heterogeneity on classification accuracy. We propose a generative learning-based personalized FL framework to enhance the privacy protection and classification accuracy of user data in the metaverse, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FedCGPL</i>. It decomposes the local networks of individual clients into a feature extractor, base layers, and personalization layers. The generator of conditional generative adversarial networks generates data that fit the real output of the extractor and are uploaded to the server for aggregation along with the base layers. This approach prevents the exposure of user data to the server within the metaverse. Based on FedCGPL, we propose FedCGPB, a generative learning-based and fast-converging personalized FL framework to secure data in the metaverse. It improves the convergence speed of FedCGPL by adding local batch normalization to the personalization layers. Experimental results show that compared to the state-of-the-art FL frameworks, FedCGPL achieves the highest classification accuracy. Compared with FedCGPL, FedCGPB achieves a higher level of convergence speed while maintaining similar classification performance.