Exact Layer Streaming: LoRA Fine-Tuning of an 8B Model on a 4 GB Laptop GPU

Parameter-efficient fine-tuning is bounded by a hard constraint: the frozen base model must fit in GPU memory. Layer streaming — holding only a small pool of decoder layers in VRAM and fetching the rest from host RAM on demand — removes that constraint in principle, but the published systems that stream weights during training target datacenter hardware: an H200 with 1.5 TB of host memory, or an RTX 4090 with 256 GB. Systems that do reach a 4 GB laptop either do not train, or do not stream the base; the published result for that hardware class is 1.3B, reached by projection rather than streaming. We report layer-streamed LoRA training on a 4 GB laptop GPU at two frontiers. On an RTX 3050 Laptop (4 GB, Windows) we fine-tune Llama-3.1-8B in NF4 at 119.6 tok/s with a 3.32 GB peak at 100% SM occupancy, and Qwen2.5-3B with an un-quantized bf16 base at 143 tok/s in 2.15 GB — a configuration that raises CUDA out of memory when trained resident on the same card. Measured overhead is 1.43x at 0.5B, the only size admitting a valid resident baseline. Because streaming failures are silent — a severed autograd path still yields a decreasing loss, since the upper layers keep learning — we contribute a correctness protocol whose central check is bit-exactness against a resident reference of the same numerics, verified across nine architecture families in two precisions. It caught a dispatch path that produced a 0.94 logit divergence with byte-identical weights and adapters, no crash, and a healthy-looking loss curve. We also report three measurement results independent of streaming: the cross-entropy logits term costs 14 bytes per element rather than the 6 first principles suggest; Windows/WDDM spills into host memory instead of raising OOM, so a completed run is not evidence that a configuration fits; and gradient accumulation is per-token I/O-neutral. Implementation: Soup v0.72.3, Apache-2.0 — https://github.com/MakazhanAlpamys/SoupMeasurement records, published verbatim including failures and discarded numbers — https://github.com/MakazhanAlpamys/Soup/tree/main/benchmarks