Python under load: a measurement study of JIT, runtime, native code and build-level optimisations

Deciding how to speed up a Python program means choosing among a dozen techniques, and comparing them directly requires conditions they share: one machine, one protocol, one baseline — and a statement of what each technique costs on the code it does not accelerate. This paper measures the whole menu on one machine, under one protocol, with one interpreter provenance. Six kernels — three compute-bound, three branchy and allocation-heavy — are implemented in pure Python, Cython, Numba, Codon (ahead of time and through its JIT), C (through ctypes) and Rust, with the three compute-bound kernels additionally in NumPy and a pybind11 binding; CPython 3.10–3.14 and the free-threaded build are compiled here from release tarballs with one compiler and one configure line; seven build configurations of 3.14.6 come from one source tree, including PGO with full LTO and the experimental tier-2 JIT; the PyPy 7.3.20 release binary, and Meta's Cinder fork with its CinderX extensions built from source on the same host, are measured against the same baseline. All timing is done by pyperf, so every number is a median over values from many independent processes, except where the text says otherwise; where a conclusion turns on two configurations being the same, it carries a significance test rather than a pair of rounded medians. Six results are worth stating up front. On compute-bound loops the choice among the compiled routes is worth far less than the decision to compile at all — they agree to within 4.4% on a scalar reduction and differ by 21% on a mandelbrot grid and by 2.8× on a naive matrix product, in both directions and with no consistent winner, against the 31–128× that compiling at all is worth — while what separates a 32× result from a 77× one on the same kernel is the floating-point contract, not the tool. That contract is only partly a compiler flag: disassembling every flag variant shows -O2 and -O3 emitting identical machine code on two of the three kernels and indistinguishable times on the third, while -march=native changes the code of all three and the run time in both directions — 1.25× faster on the mandelbrot grid, 1.80× slower on the matrix product. What -ffast-math does to the reduction is break a serial dependency chain, and the source can break it without the flag. What limits a compiler on irregular code is allocation rather than branching: C and Rust lead Cython and Numba by 3.7–5.3× exactly where small objects are allocated, by at most 17% on a branch-heavy byte scanner, and not at all on a graph traversal. Interpreter progress is real but unevenly distributed: 3.14 runs attribute and method operations 2.0–3.0× faster than 3.10, 3.11 is where most of that arrived, and 3.12 gives part of it back by running slower than 3.11 on 14 of our 16 operations. Free-threading returns 5.6–6.1× on eight threads for a 1.05–1.14× single-thread slowdown paid by the build rather than by the lock, and code whose hot loop is already native scales as well or better without it (7.2×). Composition is profile-directed rather than additive: on a mixed pipeline the one change that matched the stage profile gave 1.35×, while stacking all four techniques gave 1.26× at best and 0.73× under the GIL. Two rows concern the ground the rest stands on. Building the interpreter with PGO alone is worth 1.02×; the gain is in the combination, since PGO with full LTO is 1.10× and adding -march=native to it 1.12×, and the one operation that loses under all of them is attribute writes. CPython's own experimental JIT is 1.04× overall, improving 10 of our 16 operations and slowing 6 of them. On the alternative-runtime side, CinderX's JIT runs our six untyped kernels at 0.33–0.79× stock once it is allowed to compile after the interpreter has specialised. Compiled before first call, the same kernels run at 0.85–1.53×, so the sign of that result is a property of the measurement. Its Static Python adds a further 11.4× on a typed kernel (7.5× for a process that draws the slow code layout) — though those primitive types are worth having only because something compiles them: interpreted, even after we completed the specialising path that shipped unfinished, they are 5.8× slower than the boxed code they replace. The largest factor available without editing the source is PyPy, which runs our unmodified pure-Python kernels 3.2–42.8× faster but whose one ctypes call that hands a two-megabyte buffer across the boundary costs 5.3× more than CPython's while the calls that pass only integers are within 7%: "switch runtime" and "move the hot loop to C" exclude each other only where the boundary carries bulk data. All code, all pyperf result files and the figure and table generators are in the accompanying artifact; every figure and table here is regenerated from those files by the generator commands of §14.