Enhancing Pun Generation by Punishing Auto Generated Pun-ish Outputs

Puns are clever wordplays that exploit sound similarities while contrasting different meanings. Such complexpuns remain challenging to create, even with today’s advanced large language models. This study focuses on generatingJapanese juxtaposed puns while preserving the original meaning of input sentences. We propose a two-steptraining approach combining supervised fine-tuning (SFT) and Direct Preference Optimization (DPO). First, we performsupervised fine-tuning on a pre-trained language model using a specially designed pun paraphrase dataset. Eachpun sentence in the dataset is paired with a sentence of equivalent meaning that does not contain a pun. This step helpsthe model learn the style characteristic of puns. Second, we apply DPO, labeling human-written puns as preferredand model-generated puns from the SFT step as dispreferred. This contrastive learning guides the model in recognizingthe key elements that define pun-ness. Experimental results demonstrate that our method significantly enhancespun generation capabilities, evaluated using both neural network-based and rule-based metrics designed to measurepun-ness. Human evaluations confirm that puns generated by our approach exhibit higher pun-like qualities comparedto those produced solely by supervised fine-tuning, surpassing GPT-4o according to these metrics. Although our experimentsprimarily target Japanese puns, our model also exhibits some ability to generate English puns, suggestingpotential adaptability across languages. Overall, our findings highlight that integrating supervised fine-tuning withiterative Direct Preference Optimization improves the generation of pun-rich text without significantly compromisingthe original meaning.