A Smaller Ablation Count Is Not an Easier Model: What Heads-to-Damage Counts Measure When Attention Architectures Differ, Why Grouped-Query Attention, Model Family and Training Cannot Yet Be Separated From Scale, and the Matched Comparisons That Woul

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research A recurring worry in mechanistic interpretability is that circuit analysis becomes harder as language models grow. A 2026 workshop study challenged the worry by comparing Pythia, which uses standard multi-head attention, with Qwen2.5, which uses grouped-query attention, on indirect object identification, induction and factual recall: the Qwen2.5 models lost 80 percent of task performance after one to six ablations, the Pythia models after 2 to 135, and the study concluded that attention architecture predicts circuit concentration better than parameter count. In this analysis the author reviews and synthesises published results to show that the comparison does not yet measure what it is read as measuring, for four separable reasons. First, the counted unit differs between the families: in grouped-query attention one key/value head feeds six or seven query heads in these models, so a single ablation can remove half or a quarter of a layer's attention while a Pythia ablation removes one thirty-second to one twelfth of a layer. Second, five of the six Qwen2.5 IOI and factual-recall bottlenecks sit in the first or last layer (the 0.5B factual-recall unit is in layer 4), where deleting or swapping a layer is known to be catastrophic for every model family tested, and no control separates task-specific from generic damage. Third, an accuracy threshold counts margin and redundancy as well as localisation, and the study's own diagnostics show that in each of the two larger Pythia models the more diffuse condition is the more confident one. Fourth, the families differ in training data by a factor of sixty, in block design and in prompt format, and two other studies find grouped-query models from other families distributed and multi-head models concentrated. The author consolidates the measured values in Table 1 and Figure 1, decomposes a heads-to-damage count in Table 2, gives a procedure for reading cross-architecture concentration claims as Algorithm 1, and specifies the matched comparisons, several of which have already been trained by model developers, that would decide whether interpretability gets harder with size. On the current record neither the scale worry nor the architecture claim is established as a general statement; what is established is that circuit size, discovery cost, algorithm transfer and component meaning scale differently and must be reported separately.