Functional Correspondence through Dictionary Reuse

We compared the extent to which functional correspondencebetween corresponding Transformer blocks emerges or is preservedacross different training runs when atoms designatedas active during Source training are reused together with thedictionary scales 𝐿 in subsequent training, relative to a standardDense model. In this study, functional correspondencerefers to the degree to which corresponding Transformerblocks from different training runs exhibit similar functionalresponses. Across the experiments, functional correspondence wasconsistently higher when Source-active dictionary atoms andscales 𝐿 were preserved.