Federated Discrete Prompt Tuning for Language Models Using Synthetic Examples

Effective prompt tuning is critical for using generative AI models, such as large language models (LLMs) and small language models (SLMs), for domain-specific tasks. However, optimizing natural language prompts becomes challenging when the optimizer's access to training data is limited due to privacy concerns. In particular, we consider a multi-party situation where the optimizer has access to performance scores computed from the training dataset but does not have access to the dataset itself. To address this challenge, we propose a novel method for collaboratively optimizing natural language prompts in distributed, multiparty environments. Our approach extends the recent LLM-based prompt optimizer, Optimization by PROmpting (OPRO), by utilizing a synthetic dataset. Unlike existing federated learning-based prompt tuning techniques that only optimize continuous prompts as numerical vectors, our method allows multiple data owners to refine natural language prompts without exposing confidential data. Experiments on the GSM8K benchmark demonstrate task-specific performance improvements using the proposed method. This work provides a secure and practical solution for prompt tuning in privacy-sensitive, distributed settings, advancing the use of LLMs and SLMs in specialized domains.