PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation

Personalized large language models (LLMs) aim to tailor their outputs to user preferences.Recent advances in parameter-efficient finetuning (PEFT) methods have highlighted the effectiveness of adapting population-level LLMs to personalized LLMs by fine-tuning userspecific parameters with user history.However, user data is typically sparse, making it challenging to adapt LLMs to specific user patterns.To address this challenge, we propose PROgressive PERsonalization (PROPER), a novel progressive learning framework inspired by meso-level theory in social science.PROPER bridges population-level and user-level models by grouping users based on preferences and adapting LLMs in stages.It combines a Mixture-of-Experts (MoE) structure with Low Ranked Adaptation (LoRA), using a user-aware router to assign users to appropriate groups automatically.Additionally, a LoRA-aware router is proposed to facilitate the integration of individual user LoRAs with group-level LoRAs.Experimental results show that PROPER significantly outperforms SOTA models across multiple tasks, demonstrating the effectiveness of our approach.Our code is available at https://github.com/callanwu/PROPER.