Language Geometry: A Unified Geometric Framework for Understanding Large Language Models

This paper presents a unified geometric framework for understanding the working mechanisms of large language models. The core thesis is: language is not a sequence of positions, but a sequence of transformations. We reinterpret the word embedding space asfibers of a principal bundle, attention mechanisms as frame transformations, and languagegeneration as probabilistic path sampling of difference vectors. Seven tests (synthetic data,GloVe, BERT) validate the core propositions: difference vectors have low-dimensional structure (2-5 dimensions), different syntactic relations correspond to different subspaces, localsections can glue under compatibility conditions, and non-linearity emerges from twistedgluing. The paper traces the historical roots of this framework to Weaver’s (1949) slidingwindow observation and the Harris/Firth (1954/1957) distributional hypothesis, showinghow these insights receive geometric mechanisms in the present framework. We introduceExt/Tor as algebraic quantifiers of gluing obstructions, and develop the melody-harmonyanalogy for multi-head attention. A suite of visualization tools is developed, offering ageometric perspective for understanding, debugging, and validating large language models.