Semantic Axis Decomposition of Transformer Embeddings

This work presents an advanced method for interpreting sentence-transformer embeddings through semantic axis decomposition, capsule layering, and recursive reasoning. We identify top-N meaningful dimensions using Random Forests and assign human-interpretable labels such as "emotionality", "scientificness", and "question intent". Beyond flat interpretation, we introduce multi-layered capsule coordinates and define semantic floors—recursive embedding levels created via selective traversal across capsule stacks. Each floor transition is compactly approximated by two scalar parameters: (α) influence magnitude and (θ) semantic shift direction. This enables low-cost, interpretable, and modular reasoning. Floor pruning is also introduced: upper semantic floors can be deactivated when they contribute minimal effect, preserving performance without sacrificing depth. This is a conceptual and visual demonstration. Code and GUI tools to be released separately. Keywords: transformer, embeddings, interpretability, semantic floors, latent space, capsule layers, XAI, sentence-transformers, recursive reasoning Interactive prototype and source code:https://github.com/kexi-bq/embedding-explainer