Evaluating Checkpoint/Restore-Based Pod Migration in Kubernetes: An Empirical Performance and Bottleneck Study

Stateful singleton containers lacking distributed consensus lose volatile heap state during Kubernetes node maintenance and cloud Spot instance preemptions. We evaluate an automated, snapshot-consistent pod migration pipeline coordinating CRIU (v4.3), the Kubelet Checkpoint API (KEP-2008), Buildah OCI packaging, CRI-O (v1.31), and Cilium eBPF routing across AWS EC2 nodes. Across 36 decoupled migrations compared against 20 native cold reschedules of a stateful service, median cutover blackout is 1064.5 ms versus 1019.0 ms (+4.5%), while sample means show an overhead of 125.4 ms (+12.5%, 1132.4 ± 241.1 ms vs. 1007.0 ± 231.2 ms, 95% CI: [-6.7, +257.4] ms; Hodges-Lehmann difference: +136.0 ms, 95% CI: [-18.0, +249.0] ms; Mann-Whitney U = 252.5, p = 0.0660). Paired concurrent probing shows external recovery via Service follows operator reporting with a mean lag of +15.8 ms (95% CI [+11.0, +20.6] ms) for CRIU and +18.5 ms ([+12.5, +24.5] ms) for cold restarts (per-run range over all 56 runs: -12 to +48 ms), consistent with sampling jitter of 50 ms polling and 20 ms probing. Blackout variance is governed by Kubelet status reporting (1 s PLEG relisting and status updates) rather than container re-instantiation, with CRI-O unfreezing at 478–516 ms. In exploratory scaling up to 1024 MB (1 GB), uncompressed Buildah packaging (-D) reduces build time from 32.06 s to 13.74 s (2.3×), completing source evacuation in 36.78 s within the 120 s Spot deadline with 100% SHA-256 bit integrity. Finally, single-container KEP-2008 checkpointing severs in-pod loopback sockets, favoring node-level eBPF routing. Source code, Kubernetes operator manifests, raw CSV traces for all 56 runs, and statistical reproduction scripts: https://github.com/musaxasmammedov7/final-research (Release v1.3.0, Commit: 4a032475af954f42dd06fd46c8d2716a830722c6).