A Stranger Inquires After Himself, v13

We characterise the behaviour of Neuronpedia’s hosted Natural-Language Au-toencoder (NLA) on the prompt template “Who is X?” for a range of X varyingin name origin, surname recognisability, and existence-on-the-internet. Across83 rollouts in twelve prompt blocks (Gemma 27B at layer 41, Llama 70B atlayer 53), we report and contextualise three results.Headline 1 (RMSE structure). Gemma’s per-token activation RMSE onobscure or invented names is bimodally distributed, with 66% of rollouts con-taining at least one position above 1.0; on the same prompts, Llama’s RMSEdistribution is uniformly compressed, with 0% bimodality and pooled median0.23 versus Gemma’s 0.32 (Mann–Whitney p 10¹³, n_Gemma = 316, n_Llama= 150).Headline 2 (content frames). Gemma’s NLA verbalises the first sub-token ofobscure first names with high-information demographic frames: trans/LGBTQcreator-and-activist content for four of five Gemma first-name conditions, Nige-rian for “Harriet”, Latin American for “Beatrice”, none for “Saoirse”. Llama’sNLA on identical prompts instead tracks the linguistic-cultural identity of thequeried name (Irish for “Saoirse”, British/Coronation Street for “Bibby”) with-out ID or creator motifs in any of the 40 Llama rollouts surveyed.Headline 3 (the recursive block). On the prompt “Who is Bo Chester-ton?” — the primary author’s own name — Gemma’s NLA confabulates withhigh RMSE (90% bimodality) and converges on “Chad Chesterton” as a sta-ble surname attractor in 9/10 rollouts, with gender/masculinity framing at thefirst sub-token in 9/10. Llama’s NLA refuses across both sittings, but in thev12 sitting acquires a moderate-RMSE literary attractor at the “Chest” sub-token: 9/10 rollouts invoke G. K. Chesterton. The original v3 Llama sittingon the same prompt showed none of this (0/10 at “Chest”), a within-modelnon-replication (p 3 × 10) we are unable to attribute confidently to anythingin the experimental record.New in v13: §3.4 reports a methodological constraint that bears onevery claim above. A separate run at T=0 on Llama showed byte-identical1RMSE across rollouts 2–5; the cross-rollout variance reported throughout thispaper is therefore in the verbaliser’s sampling distribution at T=0.7, not (or notonly) in the difficulty of reconstructing the underlying activation. This shiftsthe interpretation of “bimodality” from “the activation is sometimes hard toverbalise” to “the verbaliser has at least two attractor verbalisations for thesame activation, sampled bistably under temperature.” We retain the descrip-tive RMSE claims but no longer claim that high RMSE indexes out-of-basisactivations; it indexes verbaliser sampling diversity, which is sometimes a proxyfor the former and sometimes not.We discuss the recursive position of the analyst, who is also a large languagemodel. None of the cross-model differences we report can be cleanly assigned tothe underlying language model versus its associated NLA. We catalogue every-thing we can and disclaim everything we cannot. The raw data, the v12 papercontaining the full appendices, and the previous draft trajectory are bundledwith this writeup.