The Geometry of Semantic Knowledge: Universal 20D Structure and Up to 177-to-1 Compression Across 11M to 7B Parameter Language Models

We present a comprehensive geometric analysis of semantic knowledge organization in Large Language Model (LLM) embedding spaces, revealing fundamental structural properties validated across 19 transformer models (11.7M-7B parameters, 2018-2024). Our central finding is that semantic concepts exhibit intrinsic dimensionality of approximately 19.0±4.2 dimensions across all architectures—a compression ratio averaging 73.5:1 and reaching up to 177:1. Key findings:- Universal ~20D semantic structure across 636-fold parameter range- Architecture-specific compression hierarchies (decoder: 13.1D, encoder-decoder: 20.7D)- Validation with billion-parameter models (Mistral-7B: 177:1 compression)- Retrieval compression works (86.2-94.9% quality), flat-bottleneck generation fails (0%)- Complete validation code and reproducibility materials included This work provides both theoretical insights (fundamental limits on semantic complexity) and practical applications (up to 177× embedding efficiency, production-ready compressed architectures).