What a 7B model learns from synthetic agent environments: a LoRA adapter on Qwen2.5-7B-Instruct for Indian UPI tasks, and where the gain stops

We fine-tuned a LoRA adapter on Qwen2.5-7B-Instruct using 2,530 generated training rows, then tested it on the public Navital Gym UPI tasks, which include the kinds it was trained on. On the 43 current public UPI tasks the untrained base solved 6 of 43 (14%, interval 7 to 27%) and the adapter, Navital Bay, solved 38 of 43 (88%, interval 76 to 95%). The two ran on the same tasks; a paired exact McNemar test gives p = 4.7e-10 (32 tasks solved only by Bay, 0 only by the base). The gain is on kinds that have training data: mandates, for which the training data has no template, stayed at 0 of 5. The gain did not carry over to GST, a different environment: 14 of 50 for the base and 11 of 50 for Bay (p = 0.51), so we found no transfer. Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5 ran on the same tasks; we did not fine-tune them on these tasks; their results are in the tables beside Bay's and are not matched on settings (section 5). One training run exists, so we make no claim about which part of the data produced which result.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24246215
Primary Topic
Domain Adaptation and Few-Shot Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

What a 7B model learns from synthetic agent environments: a LoRA adapter on Qwen2.5-7B-Instruct for Indian UPI tasks, and where the gain stops

Daksh Bathla
Zenodo (CERN European Organization for Nuclear Research)
Domain Adaptation and Few-Shot Learning
preprint

What a 7B model learns from synthetic agent environments: a LoRA adapter on Qwen2.5-7B-Instruct for Indian UPI tasks, and where the gain stops

Daksh Bathla
preprint en

Abstract

We fine-tuned a LoRA adapter on Qwen2.5-7B-Instruct using 2,530 generated training rows, then tested it on the public Navital Gym UPI tasks, which include the kinds it was trained on. On the 43 current public UPI tasks the untrained base solved 6 of 43 (14%, interval 7 to 27%) and the adapter, Navital Bay, solved 38 of 43 (88%, interval 76 to 95%). The two ran on the same tasks; a paired exact McNemar test gives p = 4.7e-10 (32 tasks solved only by Bay, 0 only by the base). The gain is on kinds that have training data: mandates, for which the training data has no template, stayed at 0 of 5. The gain did not carry over to GST, a different environment: 14 of 50 for the base and 11 of 50 for Bay (p = 0.51), so we found no transfer. Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5 ran on the same tasks; we did not fine-tune them on these tasks; their results are in the tables beside Bay's and are not matched on settings (section 5). One training run exists, so we make no claim about which part of the data produced which result.

Zenodo (CERN European Organization for Nuclear Research)
Domain Adaptation and Few-Shot Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
What a 7B model learns from synthetic agent environments: a LoRA adapter on Qwen2.5-7B-Instruct for Indian UPI tasks, and where the gain stops — Daksh Bathla · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS