Animate, or Inanimate, That Is the Question for Large Language Models

The cognitive core of human beings is closely connected to the concept of animacy, which significantly influences their memory, vision, and complex language comprehension. While animacy is reflected in language through subtle constraints on verbs and adjectives, it is also acquired and honed through non-linguistic experiences. In the same vein, we suggest that the limited capacity of LLMs to grasp natural language, particularly in relation to animacy, stems from the fact that these models are trained solely on textual data. Hence, the question this paper aims to answer arises: Can LLMs, in their digital wisdom, process animacy in a similar way to what humans would do? We then propose a systematic analysis via prompting approaches. In particular, we probe different LLMs using controlled lexical contrasts (animate vs. inanimate nouns) and narrative contexts in which typically inanimate entities behave as animate. Results reveal that, although LLMs have been trained predominantly on textual data, they exhibit human-like behavior when faced with typical animate and inanimate entities in alignment with earlier studies, specifically on seven LLMs selected from three major families—OpenAI (GPT-3.5, GPT-4), Meta (Llama2 7B, 13B, 70B), and Mistral (Mistral-7B, Mixtral). GPT models generally achieve the most consistent and human-like performance, and in some tasks, such as sentence plausibility and acceptability judgments, even surpass human baselines. Moreover, although to a lesser degree, the other models also assume comparable results. Hence, LLMs can adapt to understand unconventional situations by recognising oddities as animated without needing to interface with unspoken cognitive triggers humans rely on to break down animations.