How Small Can a Code Model Be? A Reproducible Study of One-Million-Parameter Python Synthesis with Compiler Feedback for Microcontrollers

Can a language model small enough to live in the flash memory of a microcontroller write useful code? We study this question with a fully reproducible setup: a benchmark of IoT-oriented Python function-synthesis tasks with executable unit tests and audited train/test separation, decoder-only Transformers with 0.53 M and 1.05 M parameters, and a portable INT8 inference engine that runs unchanged on a PC, in a browser and on an ESP32-S3. On unseen wordings of tasks from the training families, the 1.05 M model passes the unit tests on 67.9 % of tasks with one-shot greedy decoding (mean of 3 seeds). Signature anchoring, which forces the answer to start with the function header copied from the request, raises this to 77.5 %. A compiler-in-the-loop repair procedure (Recursive Error-Feedback, REF) gated by compile() and a static undefined-name check reaches 79.8 %. REF mainly fixes syntax: it is no better than resampling under the same budget, and removing the error message from the repair prompt does not hurt. The 0.53 M model is at least as accurate (82.4 % with anchoring, s.d. 1.3) in 588,504 bytes of flash. On task families held out from training, accuracy falls to 1–2 %, and on MBPP to zero. Million-parameter models are therefore practical, verifiable specialists for a closed catalogue of embedded utility functions, not general programmers. Code, data, trained models and ESP32 firmware: https://github.com/rohithyarramala/microcoder (archived version 1.0.0: https://doi.org/10.5281/zenodo.24187363).

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-13
DOI
https://doi.org/10.5281/zenodo.24189965
Primary Topic
Software Engineering Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

How Small Can a Code Model Be? A Reproducible Study of One-Million-Parameter Python Synthesis with Compiler Feedback for Microcontrollers

Rohith Yarramala
4 links
Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
preprint

How Small Can a Code Model Be? A Reproducible Study of One-Million-Parameter Python Synthesis with Compiler Feedback for Microcontrollers

Rohith Yarramala
preprint en

Abstract

Can a language model small enough to live in the flash memory of a microcontroller write useful code? We study this question with a fully reproducible setup: a benchmark of IoT-oriented Python function-synthesis tasks with executable unit tests and audited train/test separation, decoder-only Transformers with 0.53 M and 1.05 M parameters, and a portable INT8 inference engine that runs unchanged on a PC, in a browser and on an ESP32-S3. On unseen wordings of tasks from the training families, the 1.05 M model passes the unit tests on 67.9 % of tasks with one-shot greedy decoding (mean of 3 seeds). Signature anchoring, which forces the answer to start with the function header copied from the request, raises this to 77.5 %. A compiler-in-the-loop repair procedure (Recursive Error-Feedback, REF) gated by compile() and a static undefined-name check reaches 79.8 %. REF mainly fixes syntax: it is no better than resampling under the same budget, and removing the error message from the repair prompt does not hurt. The 0.53 M model is at least as accurate (82.4 % with anchoring, s.d. 1.3) in 588,504 bytes of flash. On task families held out from training, accuracy falls to 1–2 %, and on MBPP to zero. Million-parameter models are therefore practical, verifiable specialists for a closed catalogue of embedded utility functions, not general programmers. Code, data, trained models and ESP32 firmware: https://github.com/rohithyarramala/microcoder (archived version 1.0.0: https://doi.org/10.5281/zenodo.24187363).

Zenodo (CERN European Organization for Nuclear Research)
Defence Institute of Advanced Technology (IN)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon