Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning

In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks.However, their lack of emotion and fine-grained role awareness limits the model's ability to provide personalized and diverse interactions further.Current methods face high costs in collecting highquality annotated data for scenarios such as role-playing, and traditional human alignment methods are difficult to deploy due to the inherent diversity of model behavior in role-playing scenarios.Inspired by the alignment of models for safety behaviors through RLHF (Reinforcement Learning from Human Feedback), in this paper, we revisit model role-playing behavior from the perspective of persona alignment and propose a novel annotation-free framework named Persona-Aware Contrastive Learning (PCL) to align LLMs' behavior during roleplaying, enhancing the model's role consistency.Specifically, we first design a role chain method to encourage the model to self-question based on the role characteristics and dialogue context to adjust personality consistency.Then, we further enhance the model's role-playing strategy through iterative contrastive learning between the use of role characteristics and not.Experiments on both black-box and white-box LLMs show that LLMs equipped with PCL significantly outperform vanilla LLMs under automatic evaluation methods (CharEval & GPT-4) and human expert evaluation.