Beyond Short Names: Emoji Keyword Retrieval across Models and Bounded Causal Tests
Revised independent preprint, version 1.0.0. This preprint was revised in response to reviewer feedback from the NeurIPS 2026 workshop Interpretability as a Science (InterpScience). It is not an accepted workshop paper or a conference proceedings paper. We distinguish information readable from hidden states from its causal contribution to a specified computation. Using emoji as a controlled text-only case, we prospectively separate cross-model keyword retrieval, component-local mediation, and attention-path contributions after a readout reset. We compare raw, official-name, word-disjoint CLDR-keyword, and codepoint views of 55 emoji and 55 non-emoji Unicode symbols. Across nine prespecified model-tokenizer configurations, emoji keyword-minus-codepoint retrieval exceeds the registered 0.02 floor in unrestricted and exact four-view token-count-matched analyses. Keyword-mapping randomization passes the four-model partial-conjunction primary (p = 0.0027). Symbols show broad official-name retrieval and smaller, heterogeneous keyword effects. In Llama 3.2 3B, rank-8 subspaces learned from 128 source emoji satisfy no primary late-component joint necessity-and-sufficiency criterion on 192 held-out emoji. A separately specified optimized-subspace follow-up on 384 further emoji returns uncalibratable at both primary residual sites. Its full-state controls reveal a depth-dependent asymmetry, motivating an attention-path test. On 195 additional, prospectively specified items, glyph-position message disruption and restoration after readout-state replacement shift keyword cosine margins beyond equal-count suffix controls; all five simultaneous one-sided 95% lower bounds are positive. Absolute glyph-message disruption and restoration have zero mean top-1 effects. The results establish cross-model lexical readability and a conditional attention-message contribution. The tested rank-8 account remains unsupported. A subsequent exploratory response-scoring extension on 24 authored scenarios exposes wording and polarity failures in the proposed measures. A prospective text-only qualification on 32 new facts then yields 88/96 correct reserved direct/minimal answers, four errors, and four ties, failing its all-cases-correct criterion. Separate label mappings also fail their criterion. Contextual interpretation and output-linked causality remain unresolved. Reproducibility scope: the accompanying package contains the current manuscript, selected public inputs and aggregate evidence, a preserved historical compact archive, and all 4,640 score records for the explicit-report qualification with two saved-score validation implementations. T19-L and T21 support specified statistical replay; T19-C/T20-C provide summaries rather than complete item-level inference replay. Model weights, raw historical activation arrays, private individual author responses, and the original private repository history are excluded. Saved-score verification is not an independent model experiment or human validation. Licenses: the paper, its original figures, and the newly released project-authored data are CC BY 4.0. Project code and ordinary project documentation retain Apache 2.0. Third-party and historical materials retain their original licenses and notices; see the package's file-level license register. No checkpoint or tokenizer assets are redistributed.
Authors
- Ryo Higa
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-15
- DOI
- https://doi.org/10.5281/zenodo.23376698
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint