Contrastive Projection: Reading Transformer Internals Through Desuperposition

Subtracting the hidden states of two matched inputs at each layer and projecting through the unembedding matrix W_U produces a training-free, layer-by-layer readout of what separates them in token space. The subtraction desuperposes the residual stream: it cancels the shared content and isolates the axis of variation — features invisible to the logit lens on either input alone. We validate the method in three ways: causal injection recovers the prediction gap (z = 223–4249); dose-response confirms directional specificity; and MLP neurons whose fc1 weights detect the same features gate selectively on the contrast (67 strictly gated neurons across 18 tested contrasts, disproportionately impactful when ablated). For compositional contrasts, multi-contrast triangulation recovers token-shaped causal content that single-pair readout misses (1% → 77% for "caught a cold"). Applied to Phi-2 (2.7B), the method traces compound-noun recognition to a two-hop MLP→attention circuit, traces both the IOI and factual-recall circuits from their source token to the name-mover heads via position-resolved patching, distinguishes recall from hallucination, reveals that parametric knowledge triggers correction rather than override when contradicted, identifies scalar implicature in the residual stream, and demonstrates causally that metaphor is per-domain routing rather than a figurativity flag — across 15 semantic axes and four models. We release code and data.