Expert-Stream Passes and the PCIe Bound: A Cost Model and Measurement Study of Host-Memory-Offloaded MoE Serving on a Consumer GPU

Host-memory offloading makes large mixture-of-experts models accessible on consumer GPUs, but token counts alone do not explain their latency. We study Qwen3.8-Flash-Next serving on an RTX 4090 using expert-stream passes and PCIe time budgets as accounting units. Illustrative short-suffix observations are consistent with roughly one-pass and two-pass latency costs; a checkpoint boundary can add a complete expert pass even when little text is recomputed. A serial decode model provides retrospective conditional checks of several workload-dependent rates, while a retained case reaches only 93.9% of its model value. Cache misses measured on repetitive synthetic text cannot be substituted for those of realistic conversations. At the measured miss rate of a two-stream workload, the offload structure has a conditional link bound of approximately 23.9 tokens/s per stream. A separate same-window observational comparison records approximately 21% higher single-stream and 55% higher two-stream throughput for a hybrid treatment that changes CPU pinning, execution, quantization, and cache history. The observations neither isolate these factors nor establish output-quality equivalence. These results delimit opportunities for pass elimination, scheduling, and heterogeneous execution. Multi-GPU implications remain analytical, and the conclusions are limited to the measured platform, workloads, and state protocols.This record contains the manuscript and its reproduction package, version 1.0.1: LaTeX and BibTeX source, five vector PDF figures, de-identified aggregate CSV data, and Python scripts for regenerating the figures and recomputing the public arithmetic. The standalone paper.pdf is supplied for preview and reading; the ZIP contains the complete source/data/script package.The package reproduces aggregate figures and conditional calculations, not the original end-to-end serving experiments. Original prompts, per-request outputs, state traces, complete launch environments, private logs, and immutable runtime revisions are not released. The 28-case correctness table is an archived outcome register, not an independent correctness re-execution. Hybrid comparisons do not establish executor causality or output-quality equivalence, and multi-GPU results are analytical.Version 1.0.1 clarifies range-midpoint hybrid gains, the exact-byte conditional link bound, mixed-license metadata, and reference details. It adds no new serving measurements.License scope: the manuscript, figures, data, and documentation are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0); scripts/ and requirements.txt are licensed under the MIT License. LICENSE inside the ZIP specifies the file-level scope and excludes cited third-party works, software, and trademarks. The Zenodo record license identifies the principal manuscript/data license; it does not override the MIT scope for scripts.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-13
DOI
https://doi.org/10.5281/zenodo.23339235
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Expert-Stream Passes and the PCIe Bound: A Cost Model and Measurement Study of Host-Memory-Offloaded MoE Serving on a Consumer GPU

shangfeng liu
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
preprint

Expert-Stream Passes and the PCIe Bound: A Cost Model and Measurement Study of Host-Memory-Offloaded MoE Serving on a Consumer GPU

shangfeng liu
preprint en

Abstract

Host-memory offloading makes large mixture-of-experts models accessible on consumer GPUs, but token counts alone do not explain their latency. We study Qwen3.8-Flash-Next serving on an RTX 4090 using expert-stream passes and PCIe time budgets as accounting units. Illustrative short-suffix observations are consistent with roughly one-pass and two-pass latency costs; a checkpoint boundary can add a complete expert pass even when little text is recomputed. A serial decode model provides retrospective conditional checks of several workload-dependent rates, while a retained case reaches only 93.9% of its model value. Cache misses measured on repetitive synthetic text cannot be substituted for those of realistic conversations. At the measured miss rate of a two-stream workload, the offload structure has a conditional link bound of approximately 23.9 tokens/s per stream. A separate same-window observational comparison records approximately 21% higher single-stream and 55% higher two-stream throughput for a hybrid treatment that changes CPU pinning, execution, quantization, and cache history. The observations neither isolate these factors nor establish output-quality equivalence. These results delimit opportunities for pass elimination, scheduling, and heterogeneous execution. Multi-GPU implications remain analytical, and the conclusions are limited to the measured platform, workloads, and state protocols.This record contains the manuscript and its reproduction package, version 1.0.1: LaTeX and BibTeX source, five vector PDF figures, de-identified aggregate CSV data, and Python scripts for regenerating the figures and recomputing the public arithmetic. The standalone paper.pdf is supplied for preview and reading; the ZIP contains the complete source/data/script package.The package reproduces aggregate figures and conditional calculations, not the original end-to-end serving experiments. Original prompts, per-request outputs, state traces, complete launch environments, private logs, and immutable runtime revisions are not released. The 28-case correctness table is an archived outcome register, not an independent correctness re-execution. Hybrid comparisons do not establish executor causality or output-quality equivalence, and multi-GPU results are analytical.Version 1.0.1 clarifies range-midpoint hybrid gains, the exact-byte conditional link bound, mixed-license metadata, and reference details. It adds no new serving measurements.License scope: the manuscript, figures, data, and documentation are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0); scripts/ and requirements.txt are licensed under the MIT License. LICENSE inside the ZIP specifies the file-level scope and excludes cited third-party works, software, and trademarks. The Zenodo record license identifies the principal manuscript/data license; it does not override the MIT scope for scripts.

Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.