Migrating CakeML Type Soundness from HOL4 to Lean with LLM Agents: An Experience Report
Large language model (LLM) agents can translate formal developments between proof assistants, but a translation can compile, and even support checked proofs, while it differs in meaning from its source. We report our experience migrating the definitions and theorem statements that CakeML's type soundness theorem depends on from HOL4 to Lean 4. An agent translated and repaired the development, and a second model compared the translation with the source, with human adjudication introduced after incorrect repairs. Proving agents then attempted the theorems, with each proof audited through all its dependencies. Scripted structural checks exposed placeholders that compile and definitions without bodies. The second model found semantic errors but also proposed wrong repairs, and most of its early findings were not defects. Proof attempts exposed false statements and vacuous translations of nontrivial source theorems that review had missed. After repairs to statements and definitions, the automated prover Aristotle proved 267 of the 269 theorems with their statements unchanged, the main theorem included, using only Lean's standard axioms, and the other two translated statements were refuted. A later review of the translated library still found incorrect word-shift operations in the main theorem's dependencies, so a checked soundness proof can coexist with an unfaithful translation. From this experience we distil a migration workflow combining structural checks, human adjudication of model findings, proof audits through all dependencies, and explicit review of library dependencies, with proof-discovered defects returned to repair. We identify the claims each check supports and the semantic gaps that remain.
Authors
- Mengjin Zhuang (ORCID: https://orcid.org/0009-0000-5220-0889)
Institutions
- National University of Singapore (SG)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.5281/zenodo.24167913
- Primary Topic
- Logic, programming, and type systems
- Type
- preprint