Large Language Models May Be Interpretable-From Data Relations to Relation Manifestation
Large language models (LLMs) are widely regarded as inscrutable “black boxes.” This working hypothesis challenges that assumption from an engineer’s perspective. We argue that complexity does not equal complete uninterpretability. A productive entry point for understanding LLMs is to focus on **relations among data** and how these relations are organized and made computationally usable inside the model through representational transformations—a process we call **relation manifestation**. The paper develops three levels of reflection:1. Relations (comparisons, constraints, orderings, structures) are often more central to advanced computation than isolated data points.2. Whether an objectively existing relation can be effectively utilized depends heavily on the geometry of the representation space. Multi-layer nonlinear transformations can turn entangled, hard-to-use relations into forms more readily accessible to downstream operations.3. Model parameters primarily define geometric mapping rules; training continuously reshapes the internal organization of data representations rather than merely storing static knowledge. Building on these ideas, we propose the **Relation Manifestation Hypothesis (RMH)** and revisit scaling laws, emergent abilities, and grokking through the lens of evolving internal representations and relations. We clearly state the boundaries of the hypothesis (observability ≠ full explanation; decodability ≠ causal usage) and outline concrete experimental directions for testing it. This is an open research hypothesis and proposal intended to invite further formalization, experimentation, and critical discussion.
Authors
- Xiaoshi Li
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24358437
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint