Qualitative Analysis of Static and Dynamic Circuit-Conditioned Predictions across Transformer Models

This supplementary document presents qualitative analyses of circuit-conditioned predictions across diverse Transformer models. The analyses examine predictions from static circuits across varying sparsity levels, as well as predictions under pathway preservation and ablation of dynamic MLP, attention, and joint attention–MLP circuits. Representative cases illustrate how different circuit configurations influence token predictions and their normalized probabilities. For static circuits, the document reports the top 15 predicted tokens and their normalized probabilities as the number of retained pathways per sensor increases. These examples illustrate how target-relevant prediction patterns emerge from sparse, top-ranked pathways, alongside semantically or syntactically related alternatives. The analyses also include factual and counterfactual target predictions, providing qualitative evidence of how sparse, target-conditioned pathways influence model outputs.