Glimmer Scope: sparse autoencoders on nine layers of Muse Glimmer 30B

I trained TopK sparse autoencoders with 65,536 features each on nine layers of Muse Glimmer 30B, every third layer from 12 to 36. Across the nine, 519,383 features fire on a held-out sample of 5 million tokens, and every one of them has a label written by Muse Spark 1.3 from its strongest contexts. I tested 170,437 features by adding them back into the residual stream on neutral prompts, with Muse Spark grading the answers for concept and fluency. Steering peaks in the middle of the network: 9.7% to 11.4% of tested features steer cleanly at layers 21, 24 and 27, against 1.1% to 4.2% at the other six. Matching features between neighboring layers by how they fire together over 5,002,394 tokens shows a stable backbone: 43.3% of features belong to a thread that runs through all nine layers. The SAEs, the code, a public atlas and a live steering demo on the full-precision 30B model are released.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-04
DOI
https://doi.org/10.5281/zenodo.23952446
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Glimmer Scope: sparse autoencoders on nine layers of Muse Glimmer 30B

Robin Kroonen
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Glimmer Scope: sparse autoencoders on nine layers of Muse Glimmer 30B

Robin Kroonen
preprint en

Abstract

I trained TopK sparse autoencoders with 65,536 features each on nine layers of Muse Glimmer 30B, every third layer from 12 to 36. Across the nine, 519,383 features fire on a held-out sample of 5 million tokens, and every one of them has a label written by Muse Spark 1.3 from its strongest contexts. I tested 170,437 features by adding them back into the residual stream on neutral prompts, with Muse Spark grading the answers for concept and fluency. Steering peaks in the middle of the network: 9.7% to 11.4% of tested features steer cleanly at layers 21, 24 and 27, against 1.1% to 4.2% at the other six. Matching features between neighboring layers by how they fire together over 5,002,394 tokens shows a stable backbone: 43.3% of features belong to a thread that runs through all nine layers. The SAEs, the code, a public atlas and a live steering demo on the full-precision 30B model are released.

Zenodo (CERN European Organization for Nuclear Research)
KRI (JP)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon