Can LLMs Reason with Logic?

ChatGPT, a chat interface to OpenAI’s Large Language Model (LLM), amazed users after its release in 2022 with its functionality. However, LLMs are prone to “hallucinations”, which sparked lively debate about LLM reasoning capabilities, which is also the focus of this study. We chose logical inferencing, which is the cognitive process of drawing conclusions from logical assertions. We utilised computational ontologies, which, because of the logical frameworks they employ, provide a comparable reasoning baseline. We developed an ontology with an inconsistent concept, and we provided the same assertions as a zero-shot prompt for several LLMs. Even though all the LLMs provided quite an extensive explanation for their answers, all but two of them failed the basic logical inferencing experiment. The results indicate that reasoning remains a challenge for LLMs but that some of the latest LLMs have progressed significantly in their logical reasoning capabilities.