Large Language Model Based Emotion Recognition of Memes

Chinese memes are distinguished by their broad cultural coverage and the innovative use of unconventional imagery paired with witty, humorous texts. This unique combination not only entertains but also introduces multiple layers of semantic complexity. Consequently, the nuanced, culturally embedded messages in these memes are prone to misinterpretation, posing significant challenges to accurately conveying their intended emotional and cognitive meanings. Such misinterpretations can lead to cultural disconnects and impede effective communication across diverse audiences.This paper investigates whether current multimodal large models can effectively integrate the visual and textual meanings found in memes. By constructing a Chinese meme dataset, we compare the performance differences between Chinese-based and English-based multimodal large language models in a cross-linguistic environment. Furthermore, we examine the impact of prompt design on the outputs of these multimodal models.