From Model Inspection to Executable Learning Rules: A Survey of LLM-Driven Algorithm Synthesis and Validation
Learning systems run on rules that are fixed before they start: an optimizer and its schedule, an aggregation formula in federated learning, a detector for a monitored plant. Large language models (LLMs) that can read numerical state and write code now make it possible to inspect a process while it runs and to replace such a rule with a program written for that process. This survey examines that possibility along a five-stage loop of observation, agentic reasoning, program synthesis, verification and deployment. Methods are compared on four axes that cut across applications: the knowledge available to the LLM, the artifact it generates or modifies, the mechanism that validates it, and the cost and risk of using it. Three systems anchor the analysis: GradReason, which replaces training epochs with training-data-free weight updates; FedReason, which writes aggregation programs after reading client models; and PIA, which composes physical residual detectors for cyber-physical systems. They are read beside reactive optimizers, LLM-guided program search, learned optimizers, and non-agentic forms of knowledge reuse such as model annealing and knowledge fermentation. In the reported ablations, removing cheap empirical feedback costs far more than withholding model internals or documentation; agents mostly select and tune techniques that were known before their training cut-off; and once a working update is found, a fixed rule can recover most of the gain without inference cost. We derive a break-even condition for agent latency, propose a minimum evaluation protocol with budget-matched and rule-encoding controls, and set out open problems in discovery beyond prior knowledge, acceptance rules with long horizons, and synthesis under adversaries.
Authors
- Ali Mohammadi Ruzbahani (ORCID: https://orcid.org/0009-0004-3287-552X)
- Abouzadeh Rezvan
Institutions
- University of Calgary (CA)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24234920
- Primary Topic
- Privacy-Preserving Technologies in Data
- Type
- article
- Field-Weighted Citation Impact
- 0.00