Reinforcement Learning-Based Dual-Identity Double Auction in Personalized Federated Learning

Federated learning participants have two identities: model trainers and model users. As model users, participants care most about the performance of the final model on their own distributions, which is called Personal Model Performance (PMP). This makes training a single global model to accommodate all participants impractical because the data distributions of participants are heterogeneous. As model trainers, due to high training costs, participants are reluctant to contribute models if incentives are not enough. With the combination of the above two reasons, we propose a dual-identity double auction as an incentive mechanism in personalized federated learning, allowing directional selection between model users and model trainers, both of which are served by FL participants. Within the double auction framework, we devise a reinforcement learning-based model selection method. This method selects a set of models for each buyer to bid on. The bought models are aggregated to be a personalized model to achieve higher PMP. Additionally, we implement a transaction partition-based approach for determining clearing prices and winning pairs. We address the challenge of the unavailability of private yet essential data distribution information, the coupled influence of model selection and auction results on PMP, and more utility improvement ways of multi-demand dual-identity participants. Finally, our double auction optimizes the PMP of all participants and ensures the truthfulness of multi-demand dual-identity participants, which is harder compared with single-demand single-identity participants.