Validation-Selected Linear Interpolation Between Qwen2.5-0.5B Base and Instruct: Does It Beat Both Parents on ARC, HellaSwag and WinoGrande?

Interpolating the weights of a pretrained model and its fine-tuned sibling has beaten both endpoints in some vision settings. We ask whether the same holds for a small language model. We interpolate linearly between Qwen2.5-0.5B and Qwen2.5-0.5B-Instruct on eleven coefficients, choose t^* on 300 validation items per task, and report it on 3,967 disjoint test items from ARC-Easy, ARC-Challenge, HellaSwag and WinoGrande. The pre-registered criterion was a paired 95% bootstrap interval for the gain over the better original model lying entirely above zero. It was not met. The selected t^*=0.7 reaches 50.90% test accuracy against 50.25% for the base model, a difference of +0.65 pp (95% CI [-0.23,+1.52], p=0.149): a tie. The Spearman rank correlation between validation and test accuracy over the eleven coefficients is ρ=-0.03 (p=0.93), so the validation split carried no detectable information about the test ranking. By this selection procedure and this test set, gains above about 1.5 pp are excluded; smaller gains are neither confirmed nor excluded. One model pair and four benchmarks limit generality.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Authors

Publication Details

Journal
Figshare
Published
2026-11-15
DOI
https://doi.org/10.6084/m9.figshare.35013979
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Validation-Selected Linear Interpolation Between Qwen2.5-0.5B Base and Instruct: Does It Beat Both Parents on ARC, HellaSwag and WinoGrande?

Dheiver Francisco Santos
Figshare
Topic Modeling
preprint

Validation-Selected Linear Interpolation Between Qwen2.5-0.5B Base and Instruct: Does It Beat Both Parents on ARC, HellaSwag and WinoGrande?

Dheiver Francisco Santos
preprint en

Abstract

Interpolating the weights of a pretrained model and its fine-tuned sibling has beaten both endpoints in some vision settings. We ask whether the same holds for a small language model. We interpolate linearly between Qwen2.5-0.5B and Qwen2.5-0.5B-Instruct on eleven coefficients, choose t^* on 300 validation items per task, and report it on 3,967 disjoint test items from ARC-Easy, ARC-Challenge, HellaSwag and WinoGrande. The pre-registered criterion was a paired 95% bootstrap interval for the gain over the better original model lying entirely above zero. It was not met. The selected t^*=0.7 reaches 50.90% test accuracy against 50.25% for the base model, a difference of +0.65 pp (95% CI [-0.23,+1.52], p=0.149): a tie. The Spearman rank correlation between validation and test accuracy over the eleven coefficients is ρ=-0.03 (p=0.93), so the validation split carried no detectable information about the test ranking. By this selection procedure and this test set, gains above about 1.5 pp are excluded; smaller gains are neither confirmed nor excluded. One model pair and four benchmarks limit generality.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Figshare
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon