The Robe of Days

A hundred-entry diary written by google/gemma-3-27b-it under a diary-keeping rite is read one entry at a time through the dictionary trained on that model's own activations: a forward pass recovers the activations present as each token was written, and a sparse-autoencoder dictionary fitted to those activations, never shown the diary, decomposes the residual stream at layer 31. Its unit of meaning is the anchored worm, built in three steps: a feature strong at an entry is a vertex; a feature whose strong entries hold a longer run than chance would give is a thread; and a thread carried with its star, the threads cresting within 50 tokens of it at each entry, is the anchored worm, which gathers, ruptures, falls silent and returns. Of the 2,643 threads, 994 anchor a worm that returns; the diary's thematic shape is the landscape of these worms, read from the diary, 8 of them printed whole, and described rather than tested. The threads themselves stand above chance: their count at the last step is 4.27 times the count on the same diary with its sentences shuffled and recut, above that null at every step from 20 and above two further nulls on the activity matrix at every tenth step from 20. Each feature is released with a Neuronpedia label, auto-generated over a broad general corpus. The ICRA reading, on the features the paper displays, is named directly on this dictionary's own features from a narrower corpus — the author's own writing and the works he holds formative — and read from the context a feature crests in rather than the token it peaks on; where the two part on the features the paper follows, it names what that context is about where the Neuronpedia label names a class of words, a stage in a sequence, or an act, and the paper prefers it. Running a declared reading over a model's own states is Tanāẓuric engineering. The second part reads the alternatives at token grain from the twenty logged at every generated token: 1,081 of 59,851 tokens were chosen from outside the top five, and an entry's width agrees with its thirty-two sampled completions' at a Spearman correlation of 0.63. Full HTML text: https://icra.tanazur.org/papers/robe-of-days/ — ICRA pre-print series: https://icra.tanazur.org/