Repairing One Step in the Weights
Qwen3.5-4B fails long multiplication at one step: taking the digit at a given place. The step is traced to one attention head's query and repaired by a rank-16 change to 256 rows of one weight matrix, checked on held-out items.
Authors
- Aziz Latipov (ORCID: https://orcid.org/0009-0003-8938-8223)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.5281/zenodo.24163235
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00