Express What You See: Can Multimodal LLMs Decode Visual Ciphers with Intuitive Semiosis Comprehension?

Bridging the gap between visual and language remains a pivotal challenge for the multimodal community.Traditional VQA benchmarks encounter a modality gap and over-reliance on language priors, whereas human cognition excels at intuitive semiosis, associating abstract visual symbols to linguistic semantics.Inspired by this neurocognitive mechanism, we focus on emojis, the visual cipher conveying abstract textual semantics.Specifically, we propose a novel task of generating abstract linguistics from emoji sequence images, where such reasoning underpins critical applications in cryptography, thus challenging MLLMs' reasoning of decoding complex semantics of visual ciphers.We introduce eWe-bench (Express What you SeE), assessing MLLMs' capability of intuitive semiosis like humans.Our data construction framework ensures high visual sensitivity and data quality, which can be extended to future data enhancement.Evaluation results on advanced MLLMs highlight critical deficiencies in visual intuitive symbolic reasoning.We believe our interesting insights for advancing visual semiosis in MLLMs will pave the way for cryptographic analysis and high-level intuitive cognition intelligence of MLLMs 1 .What is the English idiom the emojis represented in this image?