Empathetic Response Generation via Regularized Q-Learning

This study presents a new offline reinforcement learning to improve the diversity of dialogue generation while the issue of limited training data is tackled. In particular, a regularized Q-function is introduced to generate the empathetic response where the distribution shift between the learned policy and the behavior policy is penalized and the resulting out-of-distribution response is mitigated. Importantly, an emotion aware reward function is designed to guide the agent to sufficiently learn various dialogue traits and emotions for empathetic response generation. In addition, a soft sampling method is implemented to draw the action to ensure the agent to reach those candidate responses which are marginal but appropriate. The flexibility of response generation is accordingly enhanced through the reward function by considering the informative dialogue features. The experiments on the emotional conversation generation show the merit of the proposed method.