APO-PFL: Optimizing Model Aggregation in Personalized Federated Learning with Aligned PPO

Federated Learning (FL) enables decentralized model training on distributed data sources, aggregating information to enhance generalization while preserving privacy. With the growing demand for personalized AI models, Personalized Federated Learning (PFL) has become essential for adapting models to individual user preferences. However, achieving strong global generalization while maintaining effective local personalization presents a significant challenge, especially under non-i.i.d. data distributions. We propose APO-PFL (Aligned Proximal Policy Optimization Personalized Federated Learning), a novel PFL framework that leverages Proximal Policy Optimization (PPO) to align aggregation weights dynamically. APO-PFL optimizes the fusion of local models using reinforcement learning to enhance global model performance while striving to maintain effective personalized inferences. Experimental results on CNN and GPT-2 models demonstrate that APO-PFL achieves superior global generalization and outperforms baseline methods while delivering consistent personalized predictions across heterogeneous devices.