Uncovering Ontology Knowledge in Large Language Models

Ontological knowledge, which structurally represents concepts and their interrelations, is crucial for artificial intelligence to effectively mimic human cognition and reasoning. Despite prior work indicating that Large Language Models (LLMs) encode substantial ontological knowledge, the mechanisms through which these models learn and utilize such knowledge remain poorly understood. This gap in knowledge representation and interpretability is critical, as it hampers our ability to enhance and control these models. Sparse Autoencoders (SAEs) have emerged as a promising approach to demystify the internal representations within LLMs, offering new avenues for model interpretability. In this work, we harness SAEs to decompose and interpret the complex activation patterns within LLMs, focusing specifically on the encoding of ontological knowledge. Through a series of systematic experiments, we identify and manipulate key features related to ontological relationships, demonstrating the models’ sensitivity to interventions on these features. Our findings not only shed light on the black-box nature of LLMs but also pave the way for developing models with enhanced reasoning capabilities and human-like interactions. This research contributes to the broader endeavor of creating AI systems that can effectively process and reason with structured knowledge, thus expanding their applicability in diverse cognitive tasks.