Extrapolating the Instruct Task Vector of Qwen2.5-0.5B: Scaling λ Beyond 1 Under Validation-Selected Coefficients on ARC, HellaSwag, and WinoGrande

Task arithmetic adds a scaled task vector, the difference between a fine-tuned and a pre-trained model, to the pre-trained weights. We ask whether extrapolating past the fine-tuned model (λ>1) improves accuracy when λ is chosen on held-out validation data. We scale the instruct task vector of Qwen2.5-0.5B over seven values of λ up to 2 and measure macro accuracy on ARC-Easy, ARC-Challenge, HellaSwag and WinoGrande, with 1,199 validation items for selection and 3,967 disjoint test items. The pre-registered hypothesis, that some λ>1 beats λ=1, is not confirmed. The best extrapolated coefficient on validation (λ=1.5) is 0.78 percentage points (pp) below the instruct model on test, the primary contrast (95% bootstrap CI [-1.60,+0.06], p=0.070), and 0.38 pp below the base model, the secondary contrast (CI [-1.54,+0.82]). The setting λ=2 is clearly worse (-2.28 pp, CI [-3.35,-1.16]), and validation selects λ^*=0, the base model. The test peak at λ=0.75 is exploratory and indistinguishable from the instruct model. The results cover one 0.5B model pair on knowledge and reasoning benchmarks.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Authors

Publication Details

Journal
Figshare
Published
2026-11-15
DOI
https://doi.org/10.6084/m9.figshare.35013988
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Extrapolating the Instruct Task Vector of Qwen2.5-0.5B: Scaling λ Beyond 1 Under Validation-Selected Coefficients on ARC, HellaSwag, and WinoGrande

Dheiver Francisco Santos
Figshare
Topic Modeling
preprint

Extrapolating the Instruct Task Vector of Qwen2.5-0.5B: Scaling λ Beyond 1 Under Validation-Selected Coefficients on ARC, HellaSwag, and WinoGrande

Dheiver Francisco Santos
preprint en

Abstract

Task arithmetic adds a scaled task vector, the difference between a fine-tuned and a pre-trained model, to the pre-trained weights. We ask whether extrapolating past the fine-tuned model (λ>1) improves accuracy when λ is chosen on held-out validation data. We scale the instruct task vector of Qwen2.5-0.5B over seven values of λ up to 2 and measure macro accuracy on ARC-Easy, ARC-Challenge, HellaSwag and WinoGrande, with 1,199 validation items for selection and 3,967 disjoint test items. The pre-registered hypothesis, that some λ>1 beats λ=1, is not confirmed. The best extrapolated coefficient on validation (λ=1.5) is 0.78 percentage points (pp) below the instruct model on test, the primary contrast (95% bootstrap CI [-1.60,+0.06], p=0.070), and 0.38 pp below the base model, the secondary contrast (CI [-1.54,+0.82]). The setting λ=2 is clearly worse (-2.28 pp, CI [-3.35,-1.16]), and validation selects λ^*=0, the base model. The test peak at λ=0.75 is exploratory and indistinguishable from the instruct model. The results cover one 0.5B model pair on knowledge and reasoning benchmarks.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Figshare
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon