Data-driven interpretation of dimensions in an embedding language model based on a reference knowledge graph

• We propose a new method for embedding dimensions interpretation based on assigning concepts to the individual dimensions that we apply to a variety of LLMs. • Our innovation includes 1) using ontologies within a domain to extract the interpretation concepts, 2) a new metric that allows us to measure the goodness of allocation of the concepts within dimension in terms of interpretability, and 3) a thorough study of statistical distances identifying KL as the best one. • We empirically demonstrate that the quality of interpretability within equally complex LLMs is not neccesarily associated with the language model performance. • We illustrate the feasibility and applicability of our method through experiments with two ontologies, HPO and GO, and four different LLMs. As language models are used in more applications, a key problem has become clear: their numerical embeddings are hard to interpret because it is unclear how each part of the vector relates to real-world meanings in specific fields. The prevailing embedding methods are inadequate in their current state, as they are unable to effectively bridge the gap between mathematical representations and human-understandable knowledge structures. The present study proposes a novel framework that explicitly links ontology classes to specific embedding dimensions through a dual-component architecture combining a text encoder that produces the target embedding dimensions with domain knowledge graphs. The Area Under the Interpretability Curve (AUIC) metric is introduced as a means to systematically evaluate model-alignment with ontological concepts. The analysis reveals that targeted dimensional mapping enables direct interpretation of individual vector components through ontological terms. The practical applications of this framework are illustrated through case studies in biomedical contexts, demonstrating enhanced model transparency without compromising performance. This approach establishes a measurable pathway for reconciling statistical language representations with structured domain knowledge, particularly benefiting fields requiring precise concept alignment like biomedicine. The implementation is publicly available at: https://github.com/Mellandd/DEIBO .