Glimmer Scope: sparse autoencoders on nine layers of Muse Glimmer 30B
I trained TopK sparse autoencoders with 65,536 features each on nine layers of Muse Glimmer 30B, every third layer from 12 to 36. Across the nine, 519,383 features fire on a held-out sample of 5 million tokens, and every one of them has a label written by Muse Spark 1.3 from its strongest contexts. I tested 170,437 features by adding them back into the residual stream on neutral prompts, with Muse Spark grading the answers for concept and fluency. Steering peaks in the middle of the network: 9.7% to 11.4% of tested features steer cleanly at layers 21, 24 and 27, against 1.1% to 4.2% at the other six. Matching features between neighboring layers by how they fire together over 5,002,394 tokens shows a stable backbone: 43.3% of features belong to a thread that runs through all nine layers. The SAEs, the code, a public atlas and a live steering demo on the full-precision 30B model are released.
Authors
- Robin Kroonen (ORCID: https://orcid.org/0009-0004-7796-5617)
Institutions
- KRI (JP)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5281/zenodo.23952446
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint