Harvesting Idle GPU Cycles: Preemptible Batch Work Behind Interactive LLM Serving, and Co-located RL Rollout and Training

GPUs are paid for by the hour, but much of a paid GPU-hour produces no useful AI output: interactive LLM serving is provisioned for bursts and idles between them, and reinforcement-learning (RL) pipelines that place rollout generation and training on disjoint GPU pools leave each pool waiting on the other. We argue that the right figure of merit for such systems is useful AI output per paid GPU-hour, and we report measurements of two ways to raise it. First, we run preemptible batch inference in the idle capacity behind interactive serving. A reactive harvester adds +2.03M batch tokens per GPU-hour at a cost of +26–30% interactive time-to-first-token (TTFT) p95; with perfect traffic foresight (an oracle upper bound), GPU utilization rises from 51.5% to 93.1% at +7–9% TTFT p95. In a combined-system test on one H100, 64 active batch items were preempted 64 times and resumed 64 times while an interactive request returned in 0.501 s, with all 64 items completed and 0 failures. Second, as a preliminary result, co-locating RL rollout and training on the same GPUs yields 1.74–1.75× RL steps per GPU-hour versus split placement (GPU utilization ∼48% → ∼96%) on 8× A100 80GB over 3 seeds and 8 steps. We also report a negative methodological finding: a strict 0.1% loss-match correctness gate is below run-to-run noise, since two identical split runs already diverge at step 1. We describe the problem setting, evaluation methodology, and results; system internals are out of scope.