Merging Qwen2.5-0.5B-Instruct with Qwen2.5-Coder-0.5B-Instruct: Weighted Sums, TIES and DARE Fall Below Both Parents, and the Coder Task Vector Is 13 Times Larger

We test whether a three-model merge of Qwen2.5-0.5B (base), Qwen2.5-0.5B-Instruct and Qwen2.5-Coder-0.5B-Instruct beats the best original model and the best two-model merge. The pre-registered grid (weights from 0.25 up) covers weighted sums, TIES (density 0.5) and DARE (p=0.5, five seeds). Selection used 1,199 validation items; we report 3,967 disjoint test items (macro accuracy over four benchmarks). Hypothesis 4 is rejected twice within the pre-registered grid. The validation-selected weighted sum scores 0.3855, which is -11.69 pp [-13.36, -10.00] against the best original model (base, 0.5025) and -12.35 pp [-14.07, -10.67] against the best two-model merge (LERP, t=0.7). TIES and DARE fall further, and every tested merge is below the Coder alone. An exploratory measurement gives the Coder task vector 13 times the relative norm of the Instruct vector. This is consistent with the failure, but no intervention tested the cause. The Coder's initial checkpoint was assumed to be the base (not verified), and the norm ratio and cosine depend on it. One model family at 0.5B scale.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Authors

Publication Details

Journal
Figshare
Published
2026-11-15
DOI
https://doi.org/10.6084/m9.figshare.35013991
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Merging Qwen2.5-0.5B-Instruct with Qwen2.5-Coder-0.5B-Instruct: Weighted Sums, TIES and DARE Fall Below Both Parents, and the Coder Task Vector Is 13 Times Larger

Dheiver Francisco Santos
Figshare
Topic Modeling
preprint

Merging Qwen2.5-0.5B-Instruct with Qwen2.5-Coder-0.5B-Instruct: Weighted Sums, TIES and DARE Fall Below Both Parents, and the Coder Task Vector Is 13 Times Larger

Dheiver Francisco Santos
preprint en

Abstract

We test whether a three-model merge of Qwen2.5-0.5B (base), Qwen2.5-0.5B-Instruct and Qwen2.5-Coder-0.5B-Instruct beats the best original model and the best two-model merge. The pre-registered grid (weights from 0.25 up) covers weighted sums, TIES (density 0.5) and DARE (p=0.5, five seeds). Selection used 1,199 validation items; we report 3,967 disjoint test items (macro accuracy over four benchmarks). Hypothesis 4 is rejected twice within the pre-registered grid. The validation-selected weighted sum scores 0.3855, which is -11.69 pp [-13.36, -10.00] against the best original model (base, 0.5025) and -12.35 pp [-14.07, -10.67] against the best two-model merge (LERP, t=0.7). TIES and DARE fall further, and every tested merge is below the Coder alone. An exploratory measurement gives the Coder task vector 13 times the relative norm of the Instruct vector. This is consistent with the failure, but no intervention tested the cause. The Coder's initial checkpoint was assumed to be the base (not verified), and the norm ratio and cosine depend on it. One model family at 0.5B scale.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.

Figshare
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon