Chumor 2.0: Towards Better Benchmarking Chinese Humor Understanding from (Ruo Zhi Ba)

Existing humor datasets and evaluations predominantly focus on English, leaving limited resources for culturally nuanced humor in non-English languages like Chinese.To address this gap, we construct Chumor, the first and the largest Chinese humor explanation dataset.Chumor is sourced from Ruo Zhi Ba (RZB, ), a Chinese Reddit-like platform known for sharing intellectually challenging and culturally specific jokes.We test ten LLMs through direct and chain-of-thought prompting, revealing that Chumor poses significant challenges to existing LLMs, with their accuracy slightly above random and far below human.In addition, our analysis highlights that human-annotated humor explanations are significantly better than those generated by GPT-4o and ERNIE 4-turbo .We release Chumor at https://huggingface.