Expert-Stream Passes and the PCIe Bound: A Cost Model and Measurement Study of Host-Memory-Offloaded MoE Serving on a Consumer GPU
Host-memory offloading makes large mixture-of-experts models accessible on consumer GPUs, but token counts alone do not explain their latency. We study Qwen3.8-Flash-Next serving on an RTX 4090 using expert-stream passes and PCIe time budgets as accounting units. Illustrative short-suffix observations are consistent with roughly one-pass and two-pass latency costs; a checkpoint boundary can add a complete expert pass even when little text is recomputed. A serial decode model provides retrospective conditional checks of several workload-dependent rates, while a retained case reaches only 93.9% of its model value. Cache misses measured on repetitive synthetic text cannot be substituted for those of realistic conversations. At the measured miss rate of a two-stream workload, the offload structure has a conditional link bound of approximately 23.9 tokens/s per stream. A separate same-window observational comparison records approximately 21% higher single-stream and 55% higher two-stream throughput for a hybrid treatment that changes CPU pinning, execution, quantization, and cache history. The observations neither isolate these factors nor establish output-quality equivalence. These results delimit opportunities for pass elimination, scheduling, and heterogeneous execution. Multi-GPU implications remain analytical, and the conclusions are limited to the measured platform, workloads, and state protocols.This record contains the manuscript and its reproduction package, version 1.0.1: LaTeX and BibTeX source, five vector PDF figures, de-identified aggregate CSV data, and Python scripts for regenerating the figures and recomputing the public arithmetic. The standalone paper.pdf is supplied for preview and reading; the ZIP contains the complete source/data/script package.The package reproduces aggregate figures and conditional calculations, not the original end-to-end serving experiments. Original prompts, per-request outputs, state traces, complete launch environments, private logs, and immutable runtime revisions are not released. The 28-case correctness table is an archived outcome register, not an independent correctness re-execution. Hybrid comparisons do not establish executor causality or output-quality equivalence, and multi-GPU results are analytical.Version 1.0.1 clarifies range-midpoint hybrid gains, the exact-byte conditional link bound, mixed-license metadata, and reference details. It adds no new serving measurements.License scope: the manuscript, figures, data, and documentation are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0); scripts/ and requirements.txt are licensed under the MIT License. LICENSE inside the ZIP specifies the file-level scope and excludes cited third-party works, software, and trademarks. The Zenodo record license identifies the principal manuscript/data license; it does not override the MIT scope for scripts.
Authors
- shangfeng liu (ORCID: https://orcid.org/0009-0001-7846-7554)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-13
- DOI
- https://doi.org/10.5281/zenodo.23339235
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- preprint