Language models demonstrate the good-enough processing seen in humans

Comparative illusions, also called Escher sentences, are comparative sentences that appear acceptable but challenge the boundaries of comprehension. Escher sentences, illusions that are the subject of this paper, require structural reinterpretation to resolve their anomalies, making them ideal phenomena for probing the mechanisms of language processing in humans and machines. Using human behavior as a benchmark for large language models' (LLM) performance, we assessed LLMs' behavior with three methods: prompt, probability measurement, and lexical disturbance. Our results indicate that LLMs, when prompted, display "human-like'' behavior, LLMs struggled to reliably rank surprisals, and they were sensitive to both lexical and syntactic cues in Eschers. These results indicate that LLMs seem to manifest the good-enough processing seen in human cognition. Understanding and narrowing this discrepancy could result in AI systems that are more in tune with human reasoning and more interpretable, ultimately enhancing our grasp of sentence processing in humans and machines.