pFedDHPO: A Differentiable Approach for Personalized Hyperparameter Optimization in Federated Learning

Hyperparameter optimization (HPO) is crucial for federated learning (FL) performance. Given the inherent data heterogeneity across clients, recent research has focused on providing personalized hyperparameters for individual clients. However, such personalized approaches introduce exponential search complexity as the number of clients increases, significantly reducing the efficiency of existing HPO methods. To address this challenge, we propose pFedDHPO, a novel personalized HPO framework that efficiently optimizes hyperparameters in a differentiable manner. Specifically, pFedDHPO formulates personalized HPO as an optimization problem targeting joint distribution parameters within the clients’ search space and leverages gradient information from differentiable validation loss to substantially enhance the efficiency of the HPO process. Experimental results demonstrate that pFedDHPO achieves state-of-the-art performance compared to baseline methods, improving accuracy by up to 18.35% under extreme Non-IID data distributions. Additionally, the framework reduces communication overhead by 41.2% compared to conventional HPO methods, making it highly scalable for resource-constrained FL deployments.