The Deadline-Carnot Theorem: A Finite-Time Thermodynamics of Compute-Optimal Training Under a Wall-Clock Constraint

Paper supported by machine-verified Lean 4. Compute-optimal scaling laws (Kaplan et al.; Hoffmann et al., "Chinchilla") prescribe a model-size/data-size split that minimizes loss for a fixed FLOP budget, implicitly assuming the training run may take arbitrarily long wall-clock time to realize that budget. Real training is instead bound by a hard deadline: a competitive release date, a funding window, a fixed rental period on a GPU fleet. We show that once training is required to finish within a wall-clock time tau, McCandlish et al.'s gradient-noise-scale theory of the steps/examples tradeoff becomes formally an endoreversible heat engine: batch size plays the role of a throttle, the number of optimizer steps plays the role of cycle time, and there is an absolute, un-crossable wall-clock floor tau_rm floor=S_min,t_1,N below which no finite batch size — however large — can complete training. The resulting data efficiency eta(tau)equiv D_min/D_rm used(tau) takes the exact Carnot functional form eta(tau)=1-tau_rm floor/tau, with the deadline tau playing the role of a hot reservoir and tau_rm floor the role of a cold one. Pricing wall-clock time against extra data tokens and minimizing total cost yields a closed-form golden-rule deadline tau^ast=tau_rm floor+sqrtc_D D_mintau_rm floor/c_tau — a genuine Curzon–Ahlborn-flavored square-root correction, derived rather than assumed, from the same generic near-a-critical-floor optimization that produces efficiency-at-maximum-power in physical heat engines. The theorem directly explains, in direction and mechanism, a 2026 empirical finding that wall-clock-constrained optimal model size grows differently than Chinchilla's FLOP-based prediction. All results are numerically verified (21/21); for an illustrative 18-billion-parameter model the golden-rule deadline sits at approx 30 days against an approx 18-day absolute floor, at a striking approx62% Deadline Tax in extra compute. We cast the construction as a new Intelligence Bound self-application, the Stoker, close with four testable predictions, and discuss implications for AI training infrastructure, conservation-model retraining under field-season deadlines, and the canon's own thermodynamic-economics program.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-26
DOI
https://doi.org/10.5281/zenodo.21609934
Primary Topic
Advanced Thermodynamics and Statistical Mechanics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Deadline-Carnot Theorem: A Finite-Time Thermodynamics of Compute-Optimal Training Under a Wall-Clock Constraint

Justin Hart, Aristotle (Harmonic)
Zenodo (CERN European Organization for Nuclear Research)
Advanced Thermodynamics and Statistical Mechanics
preprint

The Deadline-Carnot Theorem: A Finite-Time Thermodynamics of Compute-Optimal Training Under a Wall-Clock Constraint

Justin Hart, Aristotle (Harmonic)
preprint en

Abstract

Paper supported by machine-verified Lean 4. Compute-optimal scaling laws (Kaplan et al.; Hoffmann et al., "Chinchilla") prescribe a model-size/data-size split that minimizes loss for a fixed FLOP budget, implicitly assuming the training run may take arbitrarily long wall-clock time to realize that budget. Real training is instead bound by a hard deadline: a competitive release date, a funding window, a fixed rental period on a GPU fleet. We show that once training is required to finish within a wall-clock time tau, McCandlish et al.'s gradient-noise-scale theory of the steps/examples tradeoff becomes formally an endoreversible heat engine: batch size plays the role of a throttle, the number of optimizer steps plays the role of cycle time, and there is an absolute, un-crossable wall-clock floor tau_rm floor=S_min,t_1,N below which no finite batch size — however large — can complete training. The resulting data efficiency eta(tau)equiv D_min/D_rm used(tau) takes the exact Carnot functional form eta(tau)=1-tau_rm floor/tau, with the deadline tau playing the role of a hot reservoir and tau_rm floor the role of a cold one. Pricing wall-clock time against extra data tokens and minimizing total cost yields a closed-form golden-rule deadline tau^ast=tau_rm floor+sqrtc_D D_mintau_rm floor/c_tau — a genuine Curzon–Ahlborn-flavored square-root correction, derived rather than assumed, from the same generic near-a-critical-floor optimization that produces efficiency-at-maximum-power in physical heat engines. The theorem directly explains, in direction and mechanism, a 2026 empirical finding that wall-clock-constrained optimal model size grows differently than Chinchilla's FLOP-based prediction. All results are numerically verified (21/21); for an illustrative 18-billion-parameter model the golden-rule deadline sits at approx 30 days against an approx 18-day absolute floor, at a striking approx62% Deadline Tax in extra compute. We cast the construction as a new Intelligence Bound self-application, the Stoker, close with four testable predictions, and discuss implications for AI training infrastructure, conservation-model retraining under field-season deadlines, and the canon's own thermodynamic-economics program.

Zenodo (CERN European Organization for Nuclear Research)
University of Providence (US), Laboratoire Spécification et Vérification (FR)
Advanced Thermodynamics and Statistical Mechanics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.