Replicating Validation-Selected Weight Interpolation on SmolLM2-360M Base and Instruct: ARC, HellaSwag and WinoGrande

Selecting an interpolation coefficient between a base language model and its instruct sibling is often presented as a free improvement over both. We replicate a pre-registered test of this claim on SmolLM2-360M and its instruct version, using the same protocol as for Qwen2.5-0.5B with a coarser six-point grid, selection on 300 validation items per task, and 3,967 disjoint test items. The selected t^*=0.2 scores 55.86% on test against 55.40% for the base model: +0.46 pp (95% paired bootstrap CI [-0.10,+1.02], p=0.113). The interval contains zero, so the hypothesis is not supported. Unlike the Qwen pair, the instruct model is worse than the base model on these log-likelihood benchmarks (no chat template), by -5.64 pp (95% CI [-6.89,-4.35]). Merge accuracy declines from t=0.4 to 1.0; t=0.6 is not distinguishable from the base. A small-t merge is statistically indistinguishable from the base model (no equivalence test). Two model pairs and one hardware platform limit what can be concluded.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Authors

Publication Details

Journal
Figshare
Published
2026-11-15
DOI
https://doi.org/10.6084/m9.figshare.35014000
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Replicating Validation-Selected Weight Interpolation on SmolLM2-360M Base and Instruct: ARC, HellaSwag and WinoGrande

Dheiver Francisco Santos
Figshare
Topic Modeling
preprint

Replicating Validation-Selected Weight Interpolation on SmolLM2-360M Base and Instruct: ARC, HellaSwag and WinoGrande

Dheiver Francisco Santos
preprint en

Abstract

Selecting an interpolation coefficient between a base language model and its instruct sibling is often presented as a free improvement over both. We replicate a pre-registered test of this claim on SmolLM2-360M and its instruct version, using the same protocol as for Qwen2.5-0.5B with a coarser six-point grid, selection on 300 validation items per task, and 3,967 disjoint test items. The selected t^*=0.2 scores 55.86% on test against 55.40% for the base model: +0.46 pp (95% paired bootstrap CI [-0.10,+1.02], p=0.113). The interval contains zero, so the hypothesis is not supported. Unlike the Qwen pair, the instruct model is worse than the base model on these log-likelihood benchmarks (no chat template), by -5.64 pp (95% CI [-6.89,-4.35]). Merge accuracy declines from t=0.4 to 1.0; t=0.6 is not distinguishable from the base. A small-t merge is statistically indistinguishable from the base model (no equivalence test). Two model pairs and one hardware platform limit what can be concluded.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Figshare
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon