Beyond statistical parroting: hard-coding truth into LLMs via ontologies

Large Language Models (LLMs) are powerful but prone to hallucinations and factual inconsistencies, especially in knowledge-intensive tasks. In this work, we explore the integration of structured ontological knowledge into LLM prompts as a strategy to enhance factual accuracy and reliability. Using the Pizza Ontology as a showcase, we probe how different levels of domain grounding—ranging from base prompts to ontology-informed prompts—affect the factual accuracy of LLM responses. We tested different instruction-tuned last-generation LLMs from the Qwen and Llama families, ranging from 0.5 to 72 billion parameters, on a dataset of approximately 51 questions requiring various types of reasoning. Our results show that injecting ontological axioms into prompts improves response accuracy, demonstrating that formal domain knowledge can significantly reduce hallucinations. This proof-of-concept study highlights the potential for combining symbolic approaches with LLMs and lays the groundwork for more reliable, explainable AI systems. Our codebase is available at https://github.com/HamedBabaei/OntoTruth.