Reasoning or Memorization? Investigating LLMs’ Capability in Restoring Chinese Internet Homophones

Chinese homophones, prevalent in Internet culture, introduce rich linguistic twists to challenging language models.While native speakers disambiguate them through phonological reasoning and contextual understanding, the extent to which LLMs can effectively handle this task remains unclear, as does whether they rely on similar reasoning processes or merely memorize homophone-original word pairs in training.In this paper, we propose HomoP-CN, the first Chinese Internet homophones dataset including systematic perturbations testing for evaluating LLMs' homophone restoration capabilities.With the benchmark, we investigated the influence of semantic, phonological, and graphemic features on LLMs' restoration accuracy, measured the memorization reliance levels of each model during restoration through consistency ratios under controlled perturbations, and assessed the effectiveness of various prompting strategies, including contextual cues, pinyin augmentation, few-shot learning, and thoughtchain 1 .