Migrating CakeML Type Soundness from HOL4 to Lean with LLM Agents: An Experience Report

Large language model (LLM) agents can translate formal developments between proof assistants, but a translation can compile, and even support checked proofs, while it differs in meaning from its source. We report our experience migrating the definitions and theorem statements that CakeML's type soundness theorem depends on from HOL4 to Lean 4. An agent translated and repaired the development, and a second model compared the translation with the source, with human adjudication introduced after incorrect repairs. Proving agents then attempted the theorems, with each proof audited through all its dependencies. Scripted structural checks exposed placeholders that compile and definitions without bodies. The second model found semantic errors but also proposed wrong repairs, and most of its early findings were not defects. Proof attempts exposed false statements and vacuous translations of nontrivial source theorems that review had missed. After repairs to statements and definitions, the automated prover Aristotle proved 267 of the 269 theorems with their statements unchanged, the main theorem included, using only Lean's standard axioms, and the other two translated statements were refuted. A later review of the translated library still found incorrect word-shift operations in the main theorem's dependencies, so a checked soundness proof can coexist with an unfaithful translation. From this experience we distil a migration workflow combining structural checks, human adjudication of model findings, proof audits through all dependencies, and explicit review of library dependencies, with proof-discovered defects returned to repair. We identify the claims each check supports and the semantic gaps that remain.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-12
DOI
https://doi.org/10.5281/zenodo.24167913
Primary Topic
Logic, programming, and type systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Migrating CakeML Type Soundness from HOL4 to Lean with LLM Agents: An Experience Report

Mengjin Zhuang
Zenodo (CERN European Organization for Nuclear Research)
Logic, programming, and type systems
preprint

Migrating CakeML Type Soundness from HOL4 to Lean with LLM Agents: An Experience Report

Mengjin Zhuang
preprint en

Abstract

Large language model (LLM) agents can translate formal developments between proof assistants, but a translation can compile, and even support checked proofs, while it differs in meaning from its source. We report our experience migrating the definitions and theorem statements that CakeML's type soundness theorem depends on from HOL4 to Lean 4. An agent translated and repaired the development, and a second model compared the translation with the source, with human adjudication introduced after incorrect repairs. Proving agents then attempted the theorems, with each proof audited through all its dependencies. Scripted structural checks exposed placeholders that compile and definitions without bodies. The second model found semantic errors but also proposed wrong repairs, and most of its early findings were not defects. Proof attempts exposed false statements and vacuous translations of nontrivial source theorems that review had missed. After repairs to statements and definitions, the automated prover Aristotle proved 267 of the 269 theorems with their statements unchanged, the main theorem included, using only Lean's standard axioms, and the other two translated statements were refuted. A later review of the translated library still found incorrect word-shift operations in the main theorem's dependencies, so a checked soundness proof can coexist with an unfaithful translation. From this experience we distil a migration workflow combining structural checks, human adjudication of model findings, proof audits through all dependencies, and explicit review of library dependencies, with proof-discovered defects returned to repair. We identify the claims each check supports and the semantic gaps that remain.

Zenodo (CERN European Organization for Nuclear Research)
National University of Singapore (SG)
Logic, programming, and type systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon