Semantic Axis Decomposition of Transformer Embeddings

This work introduces a novel method for interpreting sentence-transformer embeddings by identifying and labeling the most semantically meaningful latent dimensions. Using Random Forest classifiers, we extract top-N influential coordinates and assign human-interpretable meanings such as "emotionality", "scientific intent", or "question structure". The result is a semantic heatmap that shows how individual sentences activate specific dimensions of meaning. This allows researchers and practitioners to better understand what transformer-based models are encoding and how they behave. This is a conceptual and visual demonstration. Code is not included. Keywords: transformer, embeddings, interpretability, latent space, semantic axis, XAI, sentence-transformers